光合开发者社区-论坛

Loading checkpoint shards: 100%|██████████| 4/4 [00:02<00:00, 1.75it/s] It is not recommended to quantize a loaded model. The model should be instantiated under the `init_empty_weights` context manager. Error invalid device function at line 81 in file /data/jenkins_workspace/workspace/bitsandbytes@4/csrc/ops.cu 2025-07-14 10:14:59,381 WARN exited: uar (exit status 1; not expected) 2025-07-14 10:15:00,385 INFO spawned: 'uar' with pid 185 2025-07-14 10:15:05,392 INFO success: uar entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)

回复 @changhl

```

root@1265645a06c9:/data# pip list|grep dtk

bitsandbytes 0.42.0+das.opt1.dtk24043

flash-attn 2.6.1+das.opt2.dtk24043

paddlepaddle 2.6.1+das.opt1.dtk24043

torch 2.3.0+das.opt2.dtk24043

torch-scatter 2.1.2+das.opt2.dtk24043

torchvision 0.18.1+das.opt1.dtk24043

triton 2.1.0+das.opt1.dtk24043

xformers 0.0.25+das.opt1.dtk24043

root@1265645a06c9:/data# python3 -c "from src.attn_weights_inferencer import AttnInferencerBase;AttnInferencerBase(model_name_or_path='/opt/models',device='cuda',quantize=True,)"

Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:02<00:00, 1.67it/s]

It is not recommended to quantize a loaded model. The model should be instantiated under the `init_empty_weights` context manager.

Error invalid device function at line 81 in file /data/jenkins_workspace/workspace/bitsandbytes@4/csrc/ops.cu

```

这个代码就可以触发这个错误