is that right? could you tell me how to fix this error?

[LiZhangMing]

1, dowload auto-gptq , then Change the peft_utils.py in my own auto-gptq path(python path/auto_gptq/utils/peft_utils.py) with the new one(qa-lora).

2, follow the instruction of auto-gptq, I run this project as follow ： python quant_with_alpaca.py --pretrained_model_dir /home/lizhangming/.cache/huggingface/hub/models--huggyllama--llama-7b/snapshots/8416d3fefb0cb3ff5775a7b13c1692d10ff1aa16 --quantized_model_dir llama7b-quant4bit-g32 --bits 4 --group_size 32 --save_and_reload

3,Then I got three files config.json , quantize_config.json, gptq_model-4bit-32g.bin

4, Then I copy the rest file from huggingface llama7b

5,last I run CUDA_VISIBLE_DEVICES=0 HF_DATASETS_OFFLINE=1 python qalora.py --model_path AutoGPTQ/examples/quantization/llama7b-quant4bit-g32/

##########################################The I got the error#########################

[xxw11]

Hi, in our experiment, using both FP32 and Triton might cause an increase in loss. You can try the following options:

Try uninstalling Triton directly by running pip uninstall triton. The backend will switch to Torch.
Use FP16 entirely. Although this might result in some precision loss, it will be faster and less likely to cause an increase in loss.
If you have any further issues, please feel free to reach out.

[LiZhangMing]

is that right？ 清风 ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "yuhuixu1993/qa-lora" ***@***.***>; 发送时间: 2024年5月28日(星期二) 下午4:10 ***@***.***>; ***@***.******@***.***>; 主题: Re: [yuhuixu1993/qa-lora] is that right? could you tell me how to fix this error? (Issue #36) In our experiment, using both FP32 and Triton simultaneously might lead to an increase in loss. You can try the following options: Try uninstalling Triton directly using pip uninstall triton, which will switch the backend to Torch. Use FP16 entirely. Although this may result in some precision loss, it will be faster and less likely to cause an increase in loss. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>