Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat (provider/google-vertex): support prompt caching for Anthropic Claude models #4431

Merged
merged 4 commits into from
Jan 21, 2025
Merged

feat (provider/google-vertex): support prompt caching for Anthropic Claude models #4431

merged 4 commits into from
Jan 21, 2025

Conversation

shaper
Copy link
Contributor

@shaper shaper commented Jan 17, 2025 •
edited
Loading

Updated version of #4269

  • caching is out of beta, from the FAQ, "Prompt caching is now generally available...". Observed behavior:
    • Anthropic supports prompt caching without the header presence
    • Vertex requires not passing the beta header to use caching, else it fails the request with:
Unexpected value(s) `prompt-caching-2024-07-31` for the `anthropic-beta` header. Please consult our documentation at docs.anthropic.com or try again without the header.`

caffeinum and others added 2 commits January 17, 2025 12:35
@caffeinum @shaper
Add prompt caching support for Anthropic Claude models in Google Vert…
92c367c 
…ex AI SDK

Add support for prompt caching for Anthropic Claude models in the Google Vertex AI SDK.

* **`packages/google-vertex/src/anthropic/google-vertex-anthropic-messages-settings.ts`**
  - Add `cacheControl` property to `GoogleVertexAnthropicMessagesSettings` interface.

* **`packages/google-vertex/src/anthropic/google-vertex-anthropic-provider.ts`**
  - Update `transformRequestBody` function to include `cacheControl` in the request body.

* **`packages/google-vertex/README.md`**
  - Add a section about prompt caching support for Anthropic Claude models.
  - Include an example demonstrating how to enable prompt caching.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/vercel/ai?shareId=XXXX-XXXX-XXXX-XXXX).
@shaper
rm obsolete prompt-caching beta header, no need for arg-transform or …
418a1a9 
…duplicating in settings
@shaper shaper mentioned this pull request Jan 17, 2025
Closed
@shaper shaper requested a review from lgrammel January 17, 2025 21:42
@shaper shaper marked this pull request as ready for review January 17, 2025 21:42
@shaper shaper merged commit b284e2c into main Jan 21, 2025
9 checks passed
@shaper shaper deleted the shaper/pr/4269-vertex-anthropic-caching branch January 21, 2025 19:15
@shaper shaper mentioned this pull request Jan 21, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lgrammel lgrammel lgrammel approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@shaper @lgrammel @caffeinum