Added Retrying Mechanism in case of Request Rate Limit Error for MistralAIEmbeddings
#27818
+20
−4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
keenborder786
commented
Nov 1, 2024
keenborder786
commented
- Description:: In the event of a Rate Limit Error from the MistralAI server, the response JSON raises a KeyError. To address this, a simple retry mechanism has been implemented to handle cases where the request limit is exceeded.
- Issue: Cannot create MistralAI embeddings from pdf or urls #27790
dosubot
bot
added
the
size:S
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Skipped Deployment
|
dosubot
bot
added
Ɑ: embeddings
eyurtsev
reviewed
Nov 13, 2024
| batch_responses = [] | ||
|
|
||
| @retry( | ||
| retry=retry_if_exception_type(Exception), |
There was a problem hiding this comment.
Generally, retries should never be implemented on 4xx errors (except for 408 and 429). e.g., 403 should not be retried by default
- What do we do in other parts of the code? Perhaps there's a better example that can be adopted?
- What do other models do in the code base in terms of exposing the retry parameters so users can adjust? (e.g.,what if someone wants to have the first retry after 1 second rather than 30 seconds?)
There was a problem hiding this comment.
- @eyurtsev it is a 429 error therefore retrying makes sense.
- I was not able to find any related example.
- Yes I can make the wait and stop seconds as parameters.
eyurtsev
reviewed
Nov 13, 2024
| for batch in self._get_batches(texts) | ||
| ) | ||
| if response.status_code == 429: | ||
| raise Exception("Requests rate limit exceeded") |
There was a problem hiding this comment.
This code takes an exception that was good and informative and turns it into one that's a broad Exception of type Exception -- this is usually not a good pattern for exception handling. Stack trace will be partially lost, the exception type is less specific etc.
There was a problem hiding this comment.
- Their were no specific exception being raised to being with. But I can change it from a general Exception.
eyurtsev
requested changes
Nov 13, 2024
|
@eyurtsev please check now |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.