Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Configurable timeout and retry for custom endpoints #5568

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jameslamine
Copy link
Contributor

Summary

This PR adds two new configuration options for custom endpoints:

  • timeout: Sets request timeout in milliseconds (default: 10 minutes)
  • maxRetries: Sets maximum retry attempts for failed requests (default: 2)

These settings map directly to the OpenAI SDK configuration options and help prevent hanging requests and improve error handling through automatic retries.

Fixes #5567

Example Usage:

endpoints:
  custom:
    - name: "Example"
      timeout: 5000      # 5 second timeout
      maxRetries: 3      # Retry failed requests up to 3 times

Change Type

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Testing

  1. Configure a custom endpoint with timeout and maxRetries:
endpoints:
  custom:
    - name: "Test Endpoint"
      timeout: 5000
      maxRetries: 3
  1. Test scenarios:
  • Normal API calls work as expected
  • Set a low timeout (1ms) and saw error as expected

Checklist

  • I have performed a self-review of my own code
  • My changes do not introduce new warnings
  • Local unit tests pass with my changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant