Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added AI Agent Semantic Convention #1739

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

gyliu513
Copy link
Member

Fixes #1732

Changes

Please provide a brief description of the changes here.

Note: if the PR is touching an area that is not listed in the existing areas, or the area does not have sufficient domain experts coverage, the PR might be tagged as experts needed and move slowly until experts are identified.

Merge requirement checklist

@gyliu513 gyliu513 requested review from a team as code owners January 13, 2025 20:05
@gyliu513 gyliu513 marked this pull request as draft January 13, 2025 20:06
@gyliu513 gyliu513 force-pushed the agent-sc branch 8 times, most recently from e814811 to 7b7efce Compare January 15, 2025 21:06
@gyliu513 gyliu513 marked this pull request as ready for review January 15, 2025 21:12
Copy link
Contributor

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great start!

.chloggen/1732.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
docs/gen-ai/gen-ai-agent-spans.md Outdated Show resolved Hide resolved
Copy link
Member Author

@gyliu513 gyliu513 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lmolkova thanks for all of your comments, most of them are addressed now, but I still need to check how to define the tool datastore, let me check more for this.

.chloggen/1732.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
docs/gen-ai/gen-ai-agent-spans.md Outdated Show resolved Hide resolved
@gyliu513 gyliu513 force-pushed the agent-sc branch 5 times, most recently from 5628dbe to 9904399 Compare January 17, 2025 03:45
Copy link
Contributor

@lmolkova lmolkova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, it looks great, left a small comment

model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
model/gen-ai/registry.yaml Outdated Show resolved Hide resolved
model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
@github-actions github-actions bot added the enhancement New feature or request label Jan 23, 2025
Copy link
Contributor

@karthikscale3 karthikscale3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice first pass! thanks

Copy link

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I think this concentrates too much on remote agents, which AFAIK are not widely used.

I suggest create_agent should be removed until there's more evidence of the adoption of the underlying pattern.

@@ -52,14 +58,18 @@ attribute may help identify the actual system in use for `openai`.
For custom model, a custom friendly name SHOULD be used.
If none of these options apply, the `gen_ai.system` SHOULD be set to `_OTHER`.

**[4] `gen_ai.tool.type`:** Extension: A tool executed on the agent-side to directly call external APIs, bridging the gap between the agent and real-world systems. Function: A tool executed on the client-side, where the agent generates parameters for a predefined function, and the client executes the logic. Datastore: A tool used by the agent to access and query structured or unstructured external data for retrieval-augmented tasks or knowledge updates.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not clear on the what you mean by "agent-side" and "client-side". The term "agent-side" isn't used elsewhere in this document or the repo at large. I think they need defining better.

More generally, I don't think your distinction between a function and a tool is how most people would understand them.

That said, I don't have a good definition for the distinction.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My suggestion would be:

  • tools are passed to the model - tools the model can call
  • functions are invoked when a tool is called.

Most tools are functions, but not all.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, so I see now that "agent-side" is used in the google whitepaper, but again without a definition 😞 .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will add some definition as below:

  • agent-side: Agent-side operations involve actions that are performed by the agent on the server or within the agent's controlled environment. This typically includes tasks like: Executing API calls directly; Managing data through Extensions that interact with external APIs and services etc.
  • client-side: Client-side operations are actions taken on the user's end or within the client application. This includes: Executing functions that are generated by the agent but run locally. This is particularly useful in scenarios where immediate API interaction by the agent is impractical or restricted by security policies; Processing and handling function outputs, where the logic and execution of calling external APIs are handled by the client-side application, giving developers control over the data flow and reducing the load or dependency on the agent.

docs/attributes-registry/gen-ai.md Show resolved Hide resolved
docs/gen-ai/gen-ai-agent-spans.md Show resolved Hide resolved
@gyliu513
Copy link
Member Author

Hey all, thanks for the comments, we have several follow ups for this PR, I will create related PRs soon to fix.

model/gen-ai/spans.yaml Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Needs More Approval
Development

Successfully merging this pull request may close these issues.

AI Agent Semantic Convention
5 participants