GPU：K100_AI 64G *1 镜像：vllm-0.9.2-ubuntu22.04-dtk25.04.1-rc5-rocblas104381-0915-das1.6-py3.10-20250916-rc2 模型：Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8 vllm启动报错 ``` INFO 10-27 16:46:38 [__init__.py:245] Automatically detected platform rocm. INFO 10-27 16:46:43 [api_server.py:1395] vLLM API server version 0.9.2 INFO 10-27 16:46:43 [cli_args.py:325] non-default args: {'model': 'Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8'} INFO 10-27 16:46:55 [config.py:851] This model supports multiple tasks: {'generate', 'classify', 'reward', 'embed'}. Defaulting to 'generate'. `torch_dtype` is deprecated! Use `dtype` instead! INFO 10-27 16:46:55 [config.py:1488] Using max model len 262144 WARNING 10-27 16:46:55 [config.py:972] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models. INFO 10-27 16:46:55 [config.py:2301] Chunked prefill is enabled with max_num_batched_tokens=2048. INFO 10-27 16:47:02 [__init__.py:245] Automatically detected platform rocm. INFO 10-27 16:47:06 [core.py:529] Waiting for init message from front-end. INFO 10-27 16:47:06 [core.py:71] Initializing a V1 LLM engine (v0.9.2) with config: model='Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8', speculative_config=None, tokenizer='Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=262144, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=gptq, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, pooler_config=None, compilation_config={"level":3,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":512,"local_cache_dir":null} WARNING 10-27 16:47:06 [worker_base.py:42] VLLM_RANK0_NUMA is unset or set incorrectly, vllm will not bind to numa! VLLM_RANK0_NUMA = -1 INFO 10-27 16:47:06 [worker_base.py:654] ########## 1509 process(rank0) is running on CPU(s): {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127} INFO 10-27 16:47:06 [worker_base.py:655] ########## 1509 process(rank0) is running on memnode(s): {0, 1, 2, 3, 4, 5, 6, 7} WARNING: Logging before InitGoogleLogging() is written to STDERR I1027 16:47:06.830101 1509 ProcessGroupNCCL.cpp:922] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0, SPLIT_COLOR: 0, PG Name: 0 I1027 16:47:06.830143 1509 ProcessGroupNCCL.cpp:931] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 I1027 16:47:06.830756 1509 ProcessGroupNCCL.cpp:922] [PG ID 1 PG GUID 1 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0x562053878740, SPLIT_COLOR: 3389850942126204093, PG Name: 1 I1027 16:47:06.830771 1509 ProcessGroupNCCL.cpp:931] [PG ID 1 PG GUID 1 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 I1027 16:47:06.840267 1509 ProcessGroupNCCL.cpp:922] [PG ID 2 PG GUID 3 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0x562053878740, SPLIT_COLOR: 3389850942126204093, PG Name: 3 I1027 16:47:06.840301 1509 ProcessGroupNCCL.cpp:931] [PG ID 2 PG GUID 3 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 I1027 16:47:06.841568 1509 ProcessGroupNCCL.cpp:922] [PG ID 3 PG GUID 5 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0x562053878740, SPLIT_COLOR: 3389850942126204093, PG Name: 5 I1027 16:47:06.841584 1509 ProcessGroupNCCL.cpp:931] [PG ID 3 PG GUID 5 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 I1027 16:47:06.842694 1509 ProcessGroupNCCL.cpp:922] [PG ID 4 PG GUID 7 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0x562053878740, SPLIT_COLOR: 3389850942126204093, PG Name: 7 I1027 16:47:06.842713 1509 ProcessGroupNCCL.cpp:931] [PG ID 4 PG GUID 7 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 I1027 16:47:06.843737 1509 ProcessGroupNCCL.cpp:922] [PG ID 5 PG GUID 9 Rank 0] ProcessGroupNCCL initialization options: size: 1, global rank: 0, TIMEOUT(ms): 600000, USE_HIGH_PRIORITY_STREAM: 0, SPLIT_FROM: 0x562053878740, SPLIT_COLOR: 3389850942126204093, PG Name: 9 I1027 16:47:06.843756 1509 ProcessGroupNCCL.cpp:931] [PG ID 5 PG GUID 9 Rank 0] ProcessGroupNCCL environments: NCCL version: 2.18.3, TORCH_NCCL_ASYNC_ERROR_HANDLING: 3, TORCH_NCCL_DUMP_ON_TIMEOUT: 0, TORCH_NCCL_WAIT_TIMEOUT_DUMP_MILSEC: 60000, TORCH_NCCL_DESYNC_DEBUG: 0, TORCH_NCCL_ENABLE_TIMING: 0, TORCH_NCCL_BLOCKING_WAIT: 0, TORCH_DISTRIBUTED_DEBUG: OFF, TORCH_NCCL_ENABLE_MONITORING: 1, TORCH_NCCL_HEARTBEAT_TIMEOUT_SEC: 480, TORCH_NCCL_TRACE_BUFFER_SIZE: 0, TORCH_NCCL_COORD_CHECK_MILSEC: 1000, TORCH_NCCL_NAN_CHECK: 0, TORCH_NCCL_CUDA_EVENT_CACHE: 0, TORCH_NCCL_LOG_CPP_STACK_ON_UNCLEAN_SHUTDOWN: 1 INFO 10-27 16:47:06 [parallel_state.py:1077] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0 INFO 10-27 16:47:07 [gpu_model_runner.py:1788] Starting to load model Qwen3-30B-A3B-Instruct-2507-GPTQ-Int8... INFO 10-27 16:47:07 [gpu_model_runner.py:1793] Loading model from scratch... INFO 10-27 16:47:07 [rocm.py:279] Using Flash Attention backend on V1 engine. (only supports block size 64) WARNING 10-27 16:47:07 [config.py:444] MoE DP setup unable to determine quantization scheme or unsupported quantization type. This model will not run with DP enabled. ERROR 10-27 16:47:07 [core.py:589] EngineCore failed to start. ERROR 10-27 16:47:07 [core.py:589] Traceback (most recent call last): ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 580, in run_engine_core ERROR 10-27 16:47:07 [core.py:589] engine_core = EngineCoreProc(*args, **kwargs) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 407, in __init__ ERROR 10-27 16:47:07 [core.py:589] super().__init__(vllm_config, executor_class, log_stats, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 77, in __init__ ERROR 10-27 16:47:07 [core.py:589] self.model_executor = executor_class(vllm_config) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 53, in __init__ ERROR 10-27 16:47:07 [core.py:589] self._init_executor() ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 48, in _init_executor ERROR 10-27 16:47:07 [core.py:589] self.collective_rpc("load_model") ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc ERROR 10-27 16:47:07 [core.py:589] answer = run_method(self.driver_worker, method, args, kwargs) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2908, in run_method ERROR 10-27 16:47:07 [core.py:589] return func(*args, **kwargs) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 192, in load_model ERROR 10-27 16:47:07 [core.py:589] self.model_runner.load_model() ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1794, in load_model ERROR 10-27 16:47:07 [core.py:589] self.model = model_loader.load_model( ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 38, in load_model ERROR 10-27 16:47:07 [core.py:589] model = initialize_model(vllm_config=vllm_config, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/utils.py", line 67, in initialize_model ERROR 10-27 16:47:07 [core.py:589] return model_class(vllm_config=vllm_config, prefix=prefix) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 575, in __init__ ERROR 10-27 16:47:07 [core.py:589] self.model = Qwen3MoeModel(vllm_config=vllm_config, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 154, in __init__ ERROR 10-27 16:47:07 [core.py:589] old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 353, in __init__ ERROR 10-27 16:47:07 [core.py:589] self.start_layer, self.end_layer, self.layers = make_layers( ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/utils.py", line 639, in make_layers ERROR 10-27 16:47:07 [core.py:589] [PPMissingLayer() for _ in range(start_layer)] + [ ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/utils.py", line 640, in ERROR 10-27 16:47:07 [core.py:589] maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}")) ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 355, in ERROR 10-27 16:47:07 [core.py:589] lambda prefix: Qwen3MoeDecoderLayer(config=config, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 290, in __init__ ERROR 10-27 16:47:07 [core.py:589] self.mlp = Qwen3MoeSparseMoeBlock(config=config, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 119, in __init__ ERROR 10-27 16:47:07 [core.py:589] self.experts = FusedMoE(num_experts=config.num_experts, ERROR 10-27 16:47:07 [core.py:589] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/layer.py", line 782, in __init__ ERROR 10-27 16:47:07 [core.py:589] assert quant_method is not None ERROR 10-27 16:47:07 [core.py:589] AssertionError Process EngineCore_0: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 593, in run_engine_core raise e File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 580, in run_engine_core engine_core = EngineCoreProc(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 407, in __init__ super().__init__(vllm_config, executor_class, log_stats, File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 77, in __init__ self.model_executor = executor_class(vllm_config) File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 53, in __init__ self._init_executor() File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 48, in _init_executor self.collective_rpc("load_model") File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc answer = run_method(self.driver_worker, method, args, kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2908, in run_method return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 192, in load_model self.model_runner.load_model() File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1794, in load_model self.model = model_loader.load_model( File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 38, in load_model model = initialize_model(vllm_config=vllm_config, File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/utils.py", line 67, in initialize_model return model_class(vllm_config=vllm_config, prefix=prefix) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 575, in __init__ self.model = Qwen3MoeModel(vllm_config=vllm_config, File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 154, in __init__ old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 353, in __init__ self.start_layer, self.end_layer, self.layers = make_layers( File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/utils.py", line 639, in make_layers [PPMissingLayer() for _ in range(start_layer)] + [ File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/utils.py", line 640, in maybe_offload_to_cpu(layer_fn(prefix=f"{prefix}.{idx}")) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 355, in lambda prefix: Qwen3MoeDecoderLayer(config=config, File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 290, in __init__ self.mlp = Qwen3MoeSparseMoeBlock(config=config, File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen3_moe.py", line 119, in __init__ self.experts = FusedMoE(num_experts=config.num_experts, File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/layers/fused_moe/layer.py", line 782, in __init__ assert quant_method is not None AssertionError I1027 16:47:08.603150 1509 ProcessGroupNCCL.cpp:1263] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL destructor entered. W1027 16:47:08.603206 1509 ProcessGroupNCCL.cpp:1267] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator()) I1027 16:47:08.603227 1509 ProcessGroupNCCL.cpp:1247] [PG ID 0 PG GUID 0 Rank 0] Launching ProcessGroupNCCL abort asynchrounously. I1027 16:47:08.603433 1509 ProcessGroupNCCL.cpp:1133] [PG ID 0 PG GUID 0 Rank 0] future is successfully executed for: ProcessGroup abort I1027 16:47:08.603451 1509 ProcessGroupNCCL.cpp:1254] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL aborts successfully. I1027 16:47:08.603466 1509 ProcessGroupNCCL.cpp:1284] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL watchdog thread joined. I1027 16:47:08.603513 1509 ProcessGroupNCCL.cpp:1288] [PG ID 0 PG GUID 0 Rank 0] ProcessGroupNCCL heart beat monitor thread joined. Traceback (most recent call last): File "/usr/local/bin/vllm", line 8, in sys.exit(main()) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/main.py", line 65, in main args.dispatch_function(args) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/serve.py", line 55, in cmd uvloop.run(run_server(args)) File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 82, in run return loop.run_until_complete(wrapper()) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 61, in wrapper return await main File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1431, in run_server await run_server_worker(listen_address, sock, args, **uvicorn_kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1451, in run_server_worker async with build_async_engine_client(args, client_config) as engine_client: File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__ return await anext(self.gen) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client async with build_async_engine_client_from_engine_args( File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__ return await anext(self.gen) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 194, in build_async_engine_client_from_engine_args async_llm = AsyncLLM.from_vllm_config( File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/async_llm.py", line 162, in from_vllm_config return cls( File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/async_llm.py", line 124, in __init__ self.engine_core = EngineCoreClient.make_async_mp_client( File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core_client.py", line 96, in make_async_mp_client return AsyncMPClient(*client_args) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core_client.py", line 666, in __init__ super().__init__( File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core_client.py", line 403, in __init__ with launch_core_engines(vllm_config, executor_class, File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/utils.py", line 434, in launch_core_engines wait_for_engine_startup( File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/utils.py", line 484, in wait_for_engine_startup raise RuntimeError("Engine core initialization failed. " RuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {} ```