Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add client detection #832

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Add client detection #832

wants to merge 5 commits into from

Conversation

jhrozek
Copy link
Contributor

@jhrozek jhrozek commented Jan 29, 2025

  • Add client detector interface - Adds a decorator that can be added to the FastAPI handlers and detect the client from a fallback mechanism, by the user-agent, by a specific header or by a matching word in the messages. At the moment, the clients are represented as a simple enum, but in follow-up patches they will be represented by classes that can perform the changes by an interface providing callbacks from the pipeline or other places that need client-specific behaviour.
  • Use the client type when streaming the data to the client, not when executing completion - We used to special-case ollama stream generation by passing the client type to the execute_completion. Instead, let's pass the client type to the place that needs special casing using the recently introduce client type enum.
  • Use the client type when instantiating and running provider pipelines - Instead of detecting the client type again when the pipeline is being processed, let's pass the client type on instantiating the pipeline instance as a constant and replace the hardcoded client strings by just using the constants.
  • Remove get_tool_name_from_messages - This was superseded by using the client enum.
  • Remove the is_copilot flag in favor of using the autodetected client - In the copilot provider, we can hardcode the client type to copilot when instantiating the pipelines.

Adds a decorator that can be added to the FastAPI handlers and detect
the client from a fallback mechanism, by the user-agent, by a specific
header or by a matching word in the messages.

At the moment, the clients are represented as a simple enum, but in
follow-up patches they will be represented by classes that can perform
the changes by an interface providing callbacks from the pipeline or
other places that need client-specific behaviour.

Related: #830
…xecuting completion

We used to special-case ollama stream generation by passing the client
type to the execute_completion. Instead, let's pass the client type to
the place that needs special casing using the recently introduce client
type enum.

Related: #830
Instead of detecting the client type again when the pipeline is being
processed, let's pass the client type on instantiating the pipeline
instance as a constant and replace the hardcoded client strings by just
using the constants.

Related: #830
This was superseded by using the client enum.

Related: #830
In the copilot provider, we can hardcode the client type to copilot when
instantiating the pipelines.

Related: #830
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant