Skip to content

Commit

Permalink
feat (provider/fireworks): Add Fireworks provider. (#4102)
Browse files Browse the repository at this point in the history
Co-authored-by: Lars Grammel <[email protected]>
  • Loading branch information
shaper and lgrammel authored Dec 17, 2024
1 parent 1a18d18 commit ad2bf11
Show file tree
Hide file tree
Showing 28 changed files with 1,190 additions and 175 deletions.
6 changes: 6 additions & 0 deletions .changeset/unlucky-apes-explain.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@ai-sdk/openai-compatible': patch
'@ai-sdk/fireworks': patch
---

feat (provider/fireworks): Add Fireworks provider.
125 changes: 125 additions & 0 deletions content/providers/01-ai-sdk-providers/26-fireworks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
title: Fireworks
description: Learn how to use Fireworks models with the AI SDK.
---

# Fireworks Provider

[Fireworks](https://fireworks.ai/) is a platform for running and testing LLMs through their [API](https://readme.fireworks.ai/).

## Setup

The Fireworks provider is available via the `@ai-sdk/fireworks` module. You can install it with

<Tabs items={['pnpm', 'npm', 'yarn']}>
<Tab>
<Snippet text="pnpm add @ai-sdk/fireworks" dark />
</Tab>
<Tab>
<Snippet text="npm install @ai-sdk/fireworks" dark />
</Tab>
<Tab>
<Snippet text="yarn add @ai-sdk/fireworks" dark />
</Tab>
</Tabs>

## Provider Instance

You can import the default provider instance `fireworks` from `@ai-sdk/fireworks`:

```ts
import { fireworks } from '@ai-sdk/fireworks';
```

If you need a customized setup, you can import `createFireworks` from `@ai-sdk/fireworks`
and create a provider instance with your settings:

```ts
import { createFireworks } from '@ai-sdk/fireworks';

const fireworks = createFireworks({
apiKey: process.env.FIREWORKS_API_KEY ?? '',
});
```

You can use the following optional settings to customize the Fireworks provider instance:

- **baseURL** _string_

Use a different URL prefix for API calls, e.g. to use proxy servers.
The default prefix is `https://api.fireworks.ai/inference/v1`.

- **apiKey** _string_

API key that is being sent using the `Authorization` header. It defaults to
the `FIREWORKS_API_KEY` environment variable.

- **headers** _Record&lt;string,string&gt;_

Custom headers to include in the requests.

- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_

Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.

## Language Models

You can create [Fireworks models](https://fireworks.ai/models) using a provider instance.
The first argument is the model id, e.g. `accounts/fireworks/models/firefunction-v1`:

```ts
const model = fireworks('accounts/fireworks/models/firefunction-v1');
```

### Example

You can use Fireworks language models to generate text with the `generateText` function:

```ts
import { fireworks } from '@ai-sdk/fireworks';
import { generateText } from 'ai';

const { text } = await generateText({
model: fireworks('accounts/fireworks/models/firefunction-v1'),
prompt: 'Write a vegetarian lasagna recipe for 4 people.',
});
```

Fireworks language models can also be used in the `streamText` and `streamUI` functions (see [AI SDK Core](/docs/ai-sdk-core) and [AI SDK RSC](/docs/ai-sdk-rsc)).

## Completion Models

You can create models that call the Fireworks completions API using the `.completion()` factory method:

```ts
const model = fireworks.completion('accounts/fireworks/models/firefunction-v1');
```

## Embedding Models

You can create models that call the Fireworks embeddings API using the `.textEmbeddingModel()` factory method:

```ts
const model = fireworks.textEmbeddingModel(
'accounts/fireworks/models/nomic-embed-text-v1',
);
```

## Model Capabilities

| Model | Image Input | Object Generation | Tool Usage | Tool Streaming |
| ------------------------------------------------------ | ------------------- | ------------------- | ------------------- | ------------------- |
| `accounts/fireworks/models/firefunction-v2` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `accounts/fireworks/models/llama-v3p3-70b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `accounts/fireworks/models/llama-v3p1-405b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `accounts/fireworks/models/mixtral-8x7b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `accounts/fireworks/models/mixtral-8x22b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `accounts/fireworks/models/mixtral-8x7b-instruct-hf` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `accounts/fireworks/models/qwen2p5-coder-32b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Cross size={18} /> |
| `accounts/fireworks/models/qwen2p5-72b-instruct` | <Cross size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |
| `accounts/fireworks/models/qwen2-vl-72b-instruct` | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> | <Check size={18} /> |

<Note>
The table above lists popular models. Please see the [Fireworks models
page](https://fireworks.ai/models) for a full list of available models.
</Note>
71 changes: 0 additions & 71 deletions content/providers/02-openai-compatible-providers/20-fireworks.mdx

This file was deleted.

2 changes: 2 additions & 0 deletions examples/ai-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@
"@ai-sdk/anthropic": "1.0.5",
"@ai-sdk/azure": "1.0.10",
"@ai-sdk/cohere": "1.0.5",
"@ai-sdk/fireworks": "0.0.0",
"@ai-sdk/google": "1.0.11",
"@ai-sdk/google-vertex": "2.0.9",
"@ai-sdk/groq": "1.0.8",
"@ai-sdk/mistral": "1.0.5",
"@ai-sdk/openai": "1.0.8",
"@ai-sdk/openai-compatible": "0.0.6",
"@ai-sdk/provider": "1.0.2",
"@ai-sdk/togetherai": "0.0.7",
"@ai-sdk/xai": "1.0.6",
"@opentelemetry/sdk-node": "0.54.2",
Expand Down
Loading

0 comments on commit ad2bf11

Please sign in to comment.