设备:K100 2张
系统:
[root@localhost ~]# cat /etc/os-release
NAME="openEuler"
VERSION="24.03 (LTS-SP1)"
ID="openEuler"
VERSION_ID="24.03"
PRETTY_NAME="openEuler 24.03 (LTS-SP1)"
ANSI_COLOR="0;31"
驱动:6.3.3-V1.1.0a 20251104-061232
===========================================================
使用镜像 image.sourcefind.cn:5000/dcu/admin/base/vllm:0.9.2-ubuntu22.04-dtk25.04.2-tx-1210-das1.7-py3.10-20251216启动qwen3-1.7B服务。
启动阶段会失败:
root@localhost:/workspace# vllm serve /data/models/Qwen3-1.7B/ --served-model-name qwen --gpu-memory-utilization 0.3 --trust-remote-code --enforce-eager --port 8008 --max-model-len 20480 --max-seq-len-to-capture 4096 --max-num-batched-tokens 20480 --max_num_seqs 10 --enforce-eager --disable-custom-all-reduce
INFO 12-24 09:42:53 [__init__.py:245] Automatically detected platform rocm.
INFO 12-24 09:43:04 [api_server.py:1395] vLLM API server version 0.9.2
INFO 12-24 09:43:04 [cli_args.py:325] non-default args: {'port': 8008, 'model': '/data/models/Qwen3-1.7B/', 'trust_remote_code': True, 'max_model_len': 20480, 'enforce_eager': True, 'max_seq_len_to_capture': 4096, 'served_model_name': ['qwen'], 'disable_custom_all_reduce': True, 'gpu_memory_utilization': 0.3, 'max_num_batched_tokens': 20480, 'max_num_seqs': 10}
INFO 12-24 09:43:19 [config.py:850] This model supports multiple tasks: {'classify', 'generate', 'embed', 'reward'}. Defaulting to 'generate'.
INFO 12-24 09:43:19 [config.py:1488] Using max model len 20480
WARNING 12-24 09:43:19 [rocm.py:358] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
INFO 12-24 09:43:19 [api_server.py:268] Started engine process with PID 464
INFO 12-24 09:43:27 [__init__.py:245] Automatically detected platform rocm.
INFO 12-24 09:43:31 [llm_engine.py:234] Initializing a V0 LLM engine (v0.9.2) with config: model='/data/models/Qwen3-1.7B/', speculative_config=None, tokenizer='/data/models/Qwen3-1.7B/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=20480, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='xgrammar', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=None, served_model_name=qwen, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=None, chunked_prefill_enabled=False, use_async_output_proc=False, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":false,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null}, use_cached_outputs=True,
INFO 12-24 09:43:32 [rocm.py:305] Using ROCmFlashAttention backend.
WARNING 12-24 09:43:32 [worker_base.py:42] VLLM_RANK0_NUMA is unset or set incorrectly, vllm will not bind to numa! VLLM_RANK0_NUMA = -1
INFO 12-24 09:43:32 [worker_base.py:654] ########## 464 process(rank0) is running on CPU(s): {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127}
INFO 12-24 09:43:32 [worker_base.py:655] ########## 464 process(rank0) is running on memnode(s): {1, 3, 5, 7}
INFO 12-24 09:43:32 [parallel_state.py:1153] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
INFO 12-24 09:43:32 [model_runner.py:1186] Starting to load model /data/models/Qwen3-1.7B/...
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:05<00:00, 2.51s/it]
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:05<00:00, 2.51s/it]
INFO 12-24 09:43:38 [default_loader.py:272] Loading weights took 5.14 seconds
INFO 12-24 09:43:38 [model_runner.py:1218] Model loading took 3.2152 GiB and 5.454664 seconds
/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py:1088: UserWarning: 1Torch was not compiled with memory efficient attention. (Triggered internally at /home/pytorch/aten/src/ATen/native/transformers/hip/sdp_utils.cpp:663.)
sub_out = torch.nn.functional.scaled_dot_product_attention(
ERROR 12-24 09:43:42 [engine.py:476] max(): Expected reduction dim 2 to have non-zero size.
ERROR 12-24 09:43:42 [engine.py:476] Traceback (most recent call last):
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 464, in run_mp_engine
ERROR 12-24 09:43:42 [engine.py:476] engine = MQLLMEngine.from_vllm_config(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 139, in from_vllm_config
ERROR 12-24 09:43:42 [engine.py:476] return cls(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 93, in __init__
ERROR 12-24 09:43:42 [engine.py:476] self.engine = LLMEngine(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 276, in __init__
ERROR 12-24 09:43:42 [engine.py:476] self._initialize_kv_caches()
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 424, in _initialize_kv_caches
ERROR 12-24 09:43:42 [engine.py:476] self.model_executor.determine_num_available_blocks())
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 104, in determine_num_available_blocks
ERROR 12-24 09:43:42 [engine.py:476] results = self.collective_rpc("determine_num_available_blocks")
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
ERROR 12-24 09:43:42 [engine.py:476] answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2910, in run_method
ERROR 12-24 09:43:42 [engine.py:476] return func(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 12-24 09:43:42 [engine.py:476] return func(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 257, in determine_num_available_blocks
ERROR 12-24 09:43:42 [engine.py:476] self.model_runner.profile_run()
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 12-24 09:43:42 [engine.py:476] return func(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1315, in profile_run
ERROR 12-24 09:43:42 [engine.py:476] self._dummy_run(max_num_batched_tokens, max_num_seqs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1441, in _dummy_run
ERROR 12-24 09:43:42 [engine.py:476] self.execute_model(model_input, kv_caches, intermediate_tensors)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 12-24 09:43:42 [engine.py:476] return func(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1881, in execute_model
ERROR 12-24 09:43:42 [engine.py:476] hidden_or_intermediate_states = model_executable(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 12-24 09:43:42 [engine.py:476] return self._call_impl(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 12-24 09:43:42 [engine.py:476] return forward_call(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 311, in forward
ERROR 12-24 09:43:42 [engine.py:476] hidden_states = self.model(input_ids, positions, intermediate_tensors,
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 179, in __call__
ERROR 12-24 09:43:42 [engine.py:476] return self.forward(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 379, in forward
ERROR 12-24 09:43:42 [engine.py:476] hidden_states, residual = layer(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 12-24 09:43:42 [engine.py:476] return self._call_impl(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 12-24 09:43:42 [engine.py:476] return forward_call(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 224, in forward
ERROR 12-24 09:43:42 [engine.py:476] hidden_states = self.self_attn(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 12-24 09:43:42 [engine.py:476] return self._call_impl(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 12-24 09:43:42 [engine.py:476] return forward_call(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 155, in forward
ERROR 12-24 09:43:42 [engine.py:476] attn_output = self.attn(q, k, v)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 12-24 09:43:42 [engine.py:476] return self._call_impl(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 12-24 09:43:42 [engine.py:476] return forward_call(*args, **kwargs)
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 264, in forward
ERROR 12-24 09:43:42 [engine.py:476] torch.ops.vllm.unified_attention_with_output(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1116, in __call__
ERROR 12-24 09:43:42 [engine.py:476] return self._op(*args, **(kwargs or {}))
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 523, in unified_attention_with_output
ERROR 12-24 09:43:42 [engine.py:476] self.impl.forward(self,
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 851, in forward
ERROR 12-24 09:43:42 [engine.py:476] self.sdpa_attn_func(
ERROR 12-24 09:43:42 [engine.py:476] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 1088, in _sdpa_attention
ERROR 12-24 09:43:42 [engine.py:476] sub_out = torch.nn.functional.scaled_dot_product_attention(
ERROR 12-24 09:43:42 [engine.py:476] IndexError: max(): Expected reduction dim 2 to have non-zero size.
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 478, in run_mp_engine
raise e from None
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 464, in run_mp_engine
engine = MQLLMEngine.from_vllm_config(
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 139, in from_vllm_config
return cls(
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 93, in __init__
self.engine = LLMEngine(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 276, in __init__
self._initialize_kv_caches()
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 424, in _initialize_kv_caches
self.model_executor.determine_num_available_blocks())
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 104, in determine_num_available_blocks
results = self.collective_rpc("determine_num_available_blocks")
File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc
answer = run_method(self.driver_worker, method, args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2910, in run_method
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 257, in determine_num_available_blocks
self.model_runner.profile_run()
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1315, in profile_run
self._dummy_run(max_num_batched_tokens, max_num_seqs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1441, in _dummy_run
self.execute_model(model_input, kv_caches, intermediate_tensors)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1881, in execute_model
hidden_or_intermediate_states = model_executable(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 311, in forward
hidden_states = self.model(input_ids, positions, intermediate_tensors,
File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 179, in __call__
return self.forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 379, in forward
hidden_states, residual = layer(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 224, in forward
hidden_states = self.self_attn(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3.py", line 155, in forward
attn_output = self.attn(q, k, v)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 264, in forward
torch.ops.vllm.unified_attention_with_output(
File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1116, in __call__
return self._op(*args, **(kwargs or {}))
File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 523, in unified_attention_with_output
self.impl.forward(self,
File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 851, in forward
self.sdpa_attn_func(
File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 1088, in _sdpa_attention
sub_out = torch.nn.functional.scaled_dot_product_attention(
IndexError: max(): Expected reduction dim 2 to have non-zero size.
WARNING: Logging before InitGoogleLogging() is written to STDERR
W1224 09:43:43.258553 464 ProcessGroupNCCL.cpp:1279] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())
Traceback (most recent call last):
File "/usr/local/bin/vllm", line 7, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/main.py", line 65, in main
args.dispatch_function(args)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/serve.py", line 55, in cmd
uvloop.run(run_server(args))
File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 69, in run
return loop.run_until_complete(wrapper())
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 48, in wrapper
return await main
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1431, in run_server
await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1451, in run_server_worker
async with build_async_engine_client(args, client_config) as engine_client:
File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__
return await anext(self.gen)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__
return await anext(self.gen)
File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 291, in build_async_engine_client_from_engine_args
raise RuntimeError(
RuntimeError: Engine process failed to start. See stack trace for the root cause.
===========================================================
改用image.sourcefind.cn:5000/dcu/admin/base/vllm:0.8.5-ubuntu22.04-dtk25.04.1-rc5-das1.6-py3.10-20250724
服务可以正常启动,但是推理结果是乱码???
===========================================================
镜像仓中的vllm镜像,只能让K100_AI跑,K100无法使用,为什么不标明呢;

暂无数据