1
11
1
Qwen
用vllm0.9.2最新镜像,启动Qwen3-4B-Instruct-2507失败
2025-10-28 16:56:24
162
启动命令: docker run -d --name vllm2 --network=host --ipc=host --shm-size=64G --device=/dev/kfd --device=/dev/mkfd --device=/dev/dri --env HIP_VISIBLE_DEVICES=0,1 --env HF_HUB_OFFLINE=1 --env VLLM_USE_V1=1 -v /opt/hyhal:/opt/hyhal -v /root/models/:/app/models -v ./config/:/app/config --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined -p 8013:8013 image.sourcefind.cn:5000/dcu/admin/base/vllm:0.9.2-ubuntu22.04-dtk25.04.2-py3.10 python -m vllm.entrypoints.openai.api_server --served-model-name Qwen3-4B-Instruct-2507 --model /app/models/Qwen3-4B-Instruct-2507 --max-model-len 4096 --tensor-parallel-size 2 --port 8013 --dtype auto --kv-cache-dtype auto --max-num-seqs 512 --trust-remote-code --gpu-memory-utilization 0.8 报错内容: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 478, in run_mp_engine raise e from None File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 464, in run_mp_engine engine = MQLLMEngine.from_vllm_config( File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 139, in from_vllm_config return cls( File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 93, in __init__ self.engine = LLMEngine(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 276, in __init__ self._initialize_kv_caches() File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 424, in _initialize_kv_caches self.model_executor.determine_num_available_blocks()) File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 104, in determine_num_available_blocks results = self.collective_rpc("determine_num_available_blocks") File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 332, in collective_rpc return self._run_workers(method, *args, **(kwargs or {})) File "/usr/local/lib/python3.10/dist-packages/vllm/executor/mp_distributed_executor.py", line 186, in _run_workers driver_worker_output = run_method(self.driver_worker, sent_method, File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2909, in run_method return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 257, in determine_num_available_blocks self.model_runner.profile_run() File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1315, in profile_run self._dummy_run(max_num_batched_tokens, max_num_seqs) File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1441, in _dummy_run self.execute_model(model_input, kv_caches, intermediate_tensors) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1881, in execute_model hidden_or_intermediate_states = model_executable( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 311, in forward hidden_states = self.model(input_ids, positions, intermediate_tensors, File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 179, in __call__ return self.forward(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 379, in forward hidden_states, residual = layer( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 224, in forward hidden_states = self.self_attn( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 155, in forward attn_output = self.attn(q, k, v) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 253, in forward torch.ops.vllm.unified_attention_with_output( File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1116, in __call__ return self._op(*args, **(kwargs or {})) File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 503, in unified_attention_with_output self.impl.forward(self, File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 851, in forward self.sdpa_attn_func( File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 1088, in _sdpa_attention sub_out = torch.nn.functional.scaled_dot_product_attention( IndexError: max(): Expected reduction dim 2 to have non-zero size.
qwen3.6
全部评论(2)
只看楼主
中
中大胡老师
楼主
1楼
为什么会使用到V0 Engine?
2025/10/28
Z
zc
回复 @中大胡老师
请问使用什么卡跑的?DTK版本是多少?
2025/10/29
没有更多内容了
--
楼主
Level.5
0 帖子
楼主热帖
暂无数据