1
1
1
运行时
K100AI基于vllm引擎加速运行qwen2.5-14b报错
2025-01-08 20:48:36
246

当前运行环境:
python3.10.8 flash_attn-2.6.1+das.opt2.dtk24043 vllm-0.5.0+das.opt4.dtk24043 torch-2.3.0+das.opt2.dtk24043
torchvision-0.16.0+das.opt1.dtk24043

报错如下:
File "/home/cssc/anaconda3/envs/xinference1/lib/python3.10/site-packages/torch/autograd/function.py", line 598, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "/home/cssc/anaconda3/envs/xinference1/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 621, in forward
out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _flash_attn_varlen_forward(
File "/home/cssc/anaconda3/envs/xinference1/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 89, in _flash_attn_varlen_forward
out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.varlen_fwd(
RuntimeError: [address=0.0.0.0:35457, pid=149149] HIP error: invalid device function
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing AMD_SERIALIZE_KERNEL=3.
Compile with TORCH_USE_HIP_DSA  to enable device-side assertions.

全部评论(2)
只看楼主
网
网友12JXZb
2楼
相同的报错,同问
2025/1/13
Z
zhaomy
1楼
是否安装了dtk24043呢
2025/1/11
没有更多内容了
--
楼主
Level.4
0 帖子
楼主热帖
暂无数据