Support for Multi modal inputs and generations

[karthikscale3]

Area(s)

area:gen-ai

Is your change request related to a problem? Please describe.

This relates to the support for multi modal content - images/audio etc. (inputs and generation). The current GenAI spec does not have support for multi modal inputs and generations as there needs to be additional considerations around the increase the size of the span. As a result, an external blob storage is being considered for storage and hyperlinking the stored location back to the span.

The main task relates to:

• Replace content attribute in event with a similar structure to OpenAI - array of content

Some additional comments and open questions discussed during the SIG meeting:

• We should add a hook in the instrumentation (simiar to OpenLLMetry instrumentation)
• Semantic convention for references - Add the concept of Blob Reference Properties ("reference attributes" and "reference fields") to the spec #1521
• What metadata should we stored with the blob that is stored in the bucket?

Describe the solution you'd like

The main task relates to:

• Replace content attribute in event with a similar structure to OpenAI - array of content

Describe alternatives you've considered

No response

Additional context

No response

[lmolkova]

Related to #963