You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Clone the HuggingFace repository (optional)
If the model is currently hosted in a HuggingFace repository, first clone that repository to download the raw model.
Install Git LFS, verify it's installed, and then clone the model's repository:
git lfs install
git clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 model
Convert the model
Note: some model architectures require using specific convert scripts. For example, Qwen models require running convert-hf-to-gguf.py instead of convert.py
python llm/llama.cpp/convert.py ./model --outtype f16 --outfile converted.bin
Quantize the model
llm/llama.cpp/quantize converted.bin quantized.bin q4_0
But , I can't find convert.py this file.
The text was updated successfully, but these errors were encountered:
I want to use model glmchat-6b , read doc :
Clone the HuggingFace repository (optional)
If the model is currently hosted in a HuggingFace repository, first clone that repository to download the raw model.
Install Git LFS, verify it's installed, and then clone the model's repository:
git lfs install
git clone https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1 model
Convert the model
Note: some model architectures require using specific convert scripts. For example, Qwen models require running convert-hf-to-gguf.py instead of convert.py
python llm/llama.cpp/convert.py ./model --outtype f16 --outfile converted.bin
Quantize the model
llm/llama.cpp/quantize converted.bin quantized.bin q4_0
But , I can't find convert.py this file.
The text was updated successfully, but these errors were encountered: