Skip to content

Commit

Permalink
Added DeepSeekR1-distilled models to README.md.
Browse files Browse the repository at this point in the history
  • Loading branch information
kohankhaki committed Feb 13, 2025
1 parent 74d5af9 commit ad2425f
Showing 1 changed file with 12 additions and 0 deletions.
12 changes: 12 additions & 0 deletions vec_inf/models/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,18 @@ More profiling metrics coming soon!
|:----------:|:----------:|:----------:|:----------:|
| [`QwQ-32B-Preview`](https://huggingface.co/Qwen/QwQ-32B-Preview) | 2x a40 | - tokens/s | - tokens/s |

### [DeepSeek-R1: Distilled Models](https://huggingface.co/collections/deepseek-ai/deepseek-r1-678e1e131c0169c0bc89728d)

| Variant | Suggested resource allocation | Avg prompt throughput | Avg generation throughput |
|:----------:|:----------:|:----------:|:----------:|
| [`DeepSeek-R1-Distill-Llama-70B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-70B) | 4x a40 | - tokens/s | - tokens/s |
| [`DeepSeek-R1-Distill-Llama-8B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) | 1x a40 | - tokens/s | - tokens/s |
| [`DeepSeek-R1-Distill-Qwen-32B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B) | 2x a40 | - tokens/s | - tokens/s |
| [`DeepSeek-R1-Distill-Qwen-14B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B) | 1x a40 | - tokens/s | - tokens/s |
| [`DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B) | 1x a40 | - tokens/s | - tokens/s |
| [`DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) | 1x a40 | - tokens/s | - tokens/s |


## Vision Language Models

### [LLaVa-1.5](https://huggingface.co/collections/llava-hf/llava-15-65f762d5b6941db5c2ba07e0)
Expand Down

0 comments on commit ad2425f

Please sign in to comment.