``` root@bogon:/workspace# VLLM_USE_V1=0 vllm serve /soft/models/models/Qwen/Qwen3-Reranker-0.6B --host 0.0.0.0 --port 8002 --served-model-name Qwen3-Reranker-0.6B --trust-remote-code --max-model-len 4096 --api-key root@8301 INFO 03-20 09:23:06 [__init__.py:245] Automatically detected platform rocm. INFO 03-20 09:23:11 [api_server.py:1395] vLLM API server version 0.9.2 INFO 03-20 09:23:11 [cli_args.py:325] non-default args: {'host': '0.0.0.0', 'port': 8002, 'api_key': 'root@8301', 'model': '/soft/models/models/Qwen/Qwen3-Reranker-0.6B', 'trust_remote_code': True, 'max_model_len': 4096, 'served_model_name': ['Qwen3-Reranker-0.6B']} INFO 03-20 09:23:26 [config.py:850] This model supports multiple tasks: {'classify', 'generate', 'reward', 'embed'}. Defaulting to 'generate'. `torch_dtype` is deprecated! Use `dtype` instead! INFO 03-20 09:23:26 [config.py:1488] Using max model len 4096 INFO 03-20 09:23:27 [api_server.py:268] Started engine process with PID 16114 INFO 03-20 09:23:33 [__init__.py:245] Automatically detected platform rocm. INFO 03-20 09:23:37 [llm_engine.py:234] Initializing a V0 LLM engine (v0.9.2) with config: model='/soft/models/models/Qwen/Qwen3-Reranker-0.6B', speculative_config=None, tokenizer='/soft/models/models/Qwen/Qwen3-Reranker-0.6B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='xgrammar', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=None, served_model_name=Qwen3-Reranker-0.6B, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=None, chunked_prefill_enabled=False, use_async_output_proc=True, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":false,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":256,"local_cache_dir":null}, use_cached_outputs=True, INFO 03-20 09:23:38 [rocm.py:305] Using ROCmFlashAttention backend. WARNING 03-20 09:23:38 [worker_base.py:42] VLLM_RANK0_NUMA is unset or set incorrectly, vllm will not bind to numa! VLLM_RANK0_NUMA = -1 INFO 03-20 09:23:38 [worker_base.py:654] ########## 16114 process(rank0) is running on CPU(s): {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47} INFO 03-20 09:23:38 [worker_base.py:655] ########## 16114 process(rank0) is running on memnode(s): {0, 1} WARNING: Logging before InitGoogleLogging() is written to STDERR W0320 09:23:38.361647 16114 ProcessGroupGloo.cpp:715] Warning: Unable to resolve hostname to a (local) address. Using the loopback address as fallback. Manually set the network interface to bind to with GLOO_SOCKET_IFNAME. (function operator()) INFO 03-20 09:23:38 [parallel_state.py:1153] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0 INFO 03-20 09:23:38 [model_runner.py:1186] Starting to load model /soft/models/models/Qwen/Qwen3-Reranker-0.6B... Loading safetensors checkpoint shards: 0% Completed | 0/1 [00:00 sys.exit(main()) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/main.py", line 65, in main args.dispatch_function(args) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/cli/serve.py", line 55, in cmd uvloop.run(run_server(args)) File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 69, in run return loop.run_until_complete(wrapper()) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 48, in wrapper return await main File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1431, in run_server await run_server_worker(listen_address, sock, args, **uvicorn_kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1451, in run_server_worker async with build_async_engine_client(args, client_config) as engine_client: File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__ return await anext(self.gen) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client async with build_async_engine_client_from_engine_args( File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__ return await anext(self.gen) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 291, in build_async_engine_client_from_engine_args raise RuntimeError( RuntimeError: Engine process failed to start. See stack trace for the root cause. root@bogon:/workspace# root@bogon:/workspace# ```