adds support in pipeline for voyage #3424
Conversation
|
What would be the best process of getting a review here? |
| @@ -58,6 +58,14 @@ def get_embedder(self) -> BaseEmbeddingEncoder: | |||
| ) | |||
|
|
|||
| return VertexAIEmbeddingEncoder(config=VertexAIEmbeddingConfig(**kwargs)) | |||
|
|
|||
| elif self.embedding_provider == "voyage": | |||
There was a problem hiding this comment.
Please change "voyage" to "langchain-voyageai"? You can refer to: https://github.com/Unstructured-IO/unstructured/blob/main/unstructured/ingest/interfaces.py#L237
|
Hi @jeremydiba, Thank you for your contribution. We appreciate your contribution. However, I need to inform you that we have a new git repository for ingestion and this code seems to be already in there - https://github.com/Unstructured-IO/unstructured-ingest/blob/main/unstructured_ingest/v2/processes/embedder.py. The ingestion code in this repository is no longer under active development. We are not accepting further changes to this codebase at this time. Sorry for the inconvenience this may cause. If you have any questions or would like to discuss this further, please don't hesitate to reach out. |
No problem, thanks for the response! |
The following PR adds support for Voyage embedding models within the pipeline process.
Fixes Unstructured-IO/unstructured-ingest#54