4张Z100L 部署Qwen2.5-32B-Instruct
下载路径 http://113.200.138.88:18080/aimodels/qwen/Qwen2.5-32B-Instruct
使用的镜像 dtk24.04.02-py310-all:v24.04.2
启动命令 python -m vllm.entrypoints.openai.api_server --model /data/model/Qwen2.5-32B-Instruct --trust-remote-code -tp 4 --enforce-eager

报错信息：
INFO 11-07 11:40:05 model_runner.py:159] Loading model weights took 4.4519 GB

I1107 11:40:07.170846 38343 ProcessGroupNCCL.cpp:1340] NCCL_DEBUG: N/A
Unsupported conversion from bf16 to f16

UNREACHABLE executed at ../../../lib/Conversion/TritonGPUToLLVM/ElementwiseOpToLLVM.cpp:1798!
*** SIGABRT received at time=1730950816 on cpu 21 ***
PC: @ 0x7fe32bb0a9fc (unknown) pthread_kill
@ 0x7fe32bab6520 (unknown) (unknown)
[2024-11-07 11:40:16,422 E 38343 38343] logging.cc:440: *** SIGABRT received at time=1730950816 on cpu 21 ***
[2024-11-07 11:40:16,422 E 38343 38343] logging.cc:440: PC: @ 0x7fe32bb0a9fc (unknown) pthread_kill
[2024-11-07 11:40:16,422 E 38343 38343] logging.cc:440: @ 0x7fe32bab6520 (unknown) (unknown)
Fatal Python error: Aborted

Stack (most recent call first):
File "/usr/local/lib/python3.10/site-packages/triton/compiler/compiler.py", line 194 in ttgir_to_llir
File "/usr/local/lib/python3.10/site-packages/triton/third_party/hip/hip_backend.py", line 447 in <lambda> 
File "/usr/local/lib/python3.10/site-packages/triton/compiler/compiler.py", line 595 in compile
File "/usr/local/lib/python3.10/site-packages/triton/runtime/jit.py", line 559 in run
File "/usr/local/lib/python3.10/site-packages/triton/runtime/autotuner.py", line 124 in kernel_call
File "/usr/local/lib/python3.10/site-packages/triton/testing.py", line 102 in do_bench
File "/usr/local/lib/python3.10/site-packages/triton/runtime/autotuner.py", line 136 in _bench
File "/usr/local/lib/python3.10/site-packages/triton/runtime/autotuner.py", line 161 in <dictcomp> 
File "/usr/local/lib/python3.10/site-packages/triton/runtime/autotuner.py", line 161 in run
File "/usr/local/lib/python3.10/site-packages/vllm/attention/ops/flash_attn_triton_mqa_gqa.py", line 1057 in forward
File "/usr/local/lib/python3.10/site-packages/torch/autograd/function.py", line 539 in apply
File "/usr/local/lib/python3.10/site-packages/vllm/attention/ops/flash_attn_triton_mqa_gqa.py", line 1289 in flash_attn_varlen_func
File "/usr/local/lib/python3.10/site-packages/vllm/attention/backends/rocm_flash_attn.py", line 344 in forward
File "/usr/local/lib/python3.10/site-packages/vllm/attention/layer.py", line 89 in forward
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/chatglm.py", line 118 in forward
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/chatglm.py", line 219 in forward
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/chatglm.py", line 284 in forward
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl