Skip to content

Latest commit

 

History

History

examples

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
inference
inference
 
 
logits
logits
 
 
README.md
README.md
 
 

README.md

Examples

  • inference: Examples for sending inference requests
    • llm/chat_completions.py: Python example of sending chat completion requests to OpenAI compatible server
    • llm/completions.py: Python example of sending completion requests to OpenAI compatible server
    • llm/completions.sh: Bash example of sending completion requests to OpenAI compatible server, supports JSON mode
    • text_embedding/embeddings.py: Python example of sending text embedding requests to OpenAI compatible server
    • vlm/vision_completions.py: Python example of sending chat completion requests with image attached to prompt to OpenAI compatible server for vision language models
  • logits: Example for logits generation
    • logits.py: Python example of getting logits from hosted model.