DCU: Z100L 1张 操作系统版本：kylin-sp2-x86 驱动版本：rock-5.7.1-6.2.18-1.ky10.x86\_64 使用vllm部署DeepSeek-R1-Distill-Qwen-1.5B 对话报错 镜像：image.sourcefind.cn:5000/dcu/admin/base/vllm 0.9.2-ubuntu22.04-dtk25.04.1-rc5-rocblas101839-das1.6-py3.10-20250810-alpha INFO 10-14 14:47:22 [launcher.py:37] Route: /score, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /v1/score, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /v1/audio/transcriptions, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /v1/audio/translations, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /rerank, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /v1/rerank, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /v2/rerank, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /invocations, Methods: POST INFO 10-14 14:47:22 [launcher.py:37] Route: /metrics, Methods: GET INFO: Started server process [799] INFO: Waiting for application startup. INFO: Application startup complete. INFO: 127.0.0.1:32928 - "POST /v1/chat/completions HTTP/1.1" 401 Unauthorized ERROR 10-14 14:48:46 [serving_chat.py:143] Error with model object='error' message='The model `qwen2.5` does not exist.' type='NotFoundError' param=None code=404 INFO: 127.0.0.1:45360 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found ERROR 10-14 14:50:06 [serving_chat.py:143] Error with model object='error' message='The model `qwen2.5` does not exist.' type='NotFoundError' param=None code=404 INFO: 127.0.0.1:48362 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found ERROR 10-14 14:52:50 [serving_chat.py:143] Error with model object='error' message='The model `qwen2.5` does not exist.' type='NotFoundError' param=None code=404 INFO: 127.0.0.1:36156 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found ERROR 10-14 15:05:54 [serving_chat.py:143] Error with model object='error' message='The model `qwen2.5` does not exist.' type='NotFoundError' param=None code=404 INFO: 127.0.0.1:47038 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found ERROR 10-14 15:30:13 [serving_chat.py:143] Error with model object='error' message='The model `/home/models/DeepSeek-R1-Distill-Qwen-1___5B/` does not exist.' type='NotFoundError' param=None code=404 INFO: 127.0.0.1:41872 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found INFO 10-14 15:30:27 [chat_utils.py:444] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this. INFO 10-14 15:30:28 [logger.py:43] Received request chatcmpl-5180ad069ec2445c9bf8e769cd5ccd07: prompt: '｜begin▁of▁sentence｜｜User｜你好｜Assistant｜', params: SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.05, temperature=0.7, top_p=0.8, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None, extra_args=None), prompt_token_ids: None, prompt_embeds shape: None, lora_request: None, prompt_adapter_request: None. INFO 10-14 15:30:28 [async_llm.py:270] Added request chatcmpl-5180ad069ec2445c9bf8e769cd5ccd07. loc("/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py":51:0): error: unsupported target: 'gfx906' ERROR 10-14 15:30:30 [dump_input.py:69] Dumping input data for V1 LLM engine (v0.9.2) with config: model='/home/models/DeepSeek-R1-Distill-Qwen-1.5B/', speculative_config=None, tokenizer='/home/models/DeepSeek-R1-Distill-Qwen-1.5B/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=131072, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=/home/models/DeepSeek-R1-Distill-Qwen-1.5B/, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=False, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null}, ERROR 10-14 15:30:30 [dump_input.py:76] Dumping scheduler output for model execution: SchedulerOutput(scheduled_new_reqs=[NewRequestData(req_id=chatcmpl-5180ad069ec2445c9bf8e769cd5ccd07,prompt_token_ids_len=4,mm_inputs=[],mm_hashes=[],mm_positions=[],sampling_params=SamplingParams(n=1, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalty=1.05, temperature=0.7, top_p=0.8, top_k=0, min_p=0.0, seed=None, stop=[], stop_token_ids=[], bad_words=[], include_stop_str_in_output=False, ignore_eos=False, max_tokens=512, min_tokens=0, logprobs=None, prompt_logprobs=None, skip_special_tokens=True, spaces_between_special_tokens=True, truncate_prompt_tokens=None, guided_decoding=None, extra_args=None),block_ids=([1],),num_computed_tokens=0,lora_request=None)], scheduled_cached_reqs=CachedRequestData(req_ids=[], resumed_from_preemption=[], new_token_ids=[], new_block_ids=[], num_computed_tokens=[]), num_scheduled_tokens={chatcmpl-5180ad069ec2445c9bf8e769cd5ccd07: 4}, total_num_scheduled_tokens=4, scheduled_spec_decode_tokens={}, scheduled_encoder_inputs={}, num_common_prefix_blocks=[1], finished_req_ids=[], free_encoder_input_ids=[], structured_output_request_ids={}, grammar_bitmask=null, kv_connector_metadata=null) ERROR 10-14 15:30:30 [dump_input.py:79] Dumping scheduler stats: SchedulerStats(num_running_reqs=1, num_waiting_reqs=0, kv_cache_usage=0.000198925800676375, prefix_cache_stats=PrefixCacheStats(reset=False, requests=1, queries=4, hits=0), spec_decoding_stats=None, num_corrupted_reqs=0) ERROR 10-14 15:30:30 [core.py:588] EngineCore encountered a fatal error. ERROR 10-14 15:30:30 [core.py:588] Traceback (most recent call last): ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 579, in run_engine_core ERROR 10-14 15:30:30 [core.py:588] engine_core.run_busy_loop() ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 606, in run_busy_loop ERROR 10-14 15:30:30 [core.py:588] self._process_engine_step() ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 631, in _process_engine_step ERROR 10-14 15:30:30 [core.py:588] outputs, model_executed = self.step_fn() ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 235, in step ERROR 10-14 15:30:30 [core.py:588] model_output = self.execute_model(scheduler_output) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 221, in execute_model ERROR 10-14 15:30:30 [core.py:588] raise err ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 212, in execute_model ERROR 10-14 15:30:30 [core.py:588] return self.model_executor.execute_model(scheduler_output) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute_model ERROR 10-14 15:30:30 [core.py:588] output = self.collective_rpc("execute_model", ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc ERROR 10-14 15:30:30 [core.py:588] answer = run_method(self.driver_worker, method, args, kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2910, in run_method ERROR 10-14 15:30:30 [core.py:588] return func(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context ERROR 10-14 15:30:30 [core.py:588] return func(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 308, in execute_model ERROR 10-14 15:30:30 [core.py:588] output = self.model_runner.execute_model(scheduler_output, ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context ERROR 10-14 15:30:30 [core.py:588] return func(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1382, in execute_model ERROR 10-14 15:30:30 [core.py:588] model_output = self.model( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl ERROR 10-14 15:30:30 [core.py:588] return self._call_impl(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl ERROR 10-14 15:30:30 [core.py:588] return forward_call(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 564, in forward ERROR 10-14 15:30:30 [core.py:588] hidden_states = self.model(input_ids, positions, intermediate_tensors, ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 173, in __call__ ERROR 10-14 15:30:30 [core.py:588] return self.forward(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 379, in forward ERROR 10-14 15:30:30 [core.py:588] hidden_states, residual = layer( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl ERROR 10-14 15:30:30 [core.py:588] return self._call_impl(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl ERROR 10-14 15:30:30 [core.py:588] return forward_call(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 268, in forward ERROR 10-14 15:30:30 [core.py:588] hidden_states = self.self_attn( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl ERROR 10-14 15:30:30 [core.py:588] return self._call_impl(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl ERROR 10-14 15:30:30 [core.py:588] return forward_call(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 198, in forward ERROR 10-14 15:30:30 [core.py:588] attn_output = self.attn(q, k, v) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl ERROR 10-14 15:30:30 [core.py:588] return self._call_impl(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl ERROR 10-14 15:30:30 [core.py:588] return forward_call(*args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 243, in forward ERROR 10-14 15:30:30 [core.py:588] torch.ops.vllm.unified_attention_with_output( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1116, in __call__ ERROR 10-14 15:30:30 [core.py:588] return self._op(*args, **(kwargs or {})) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 452, in unified_attention_with_output ERROR 10-14 15:30:30 [core.py:588] self.impl.forward(self, ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/attention/backends/triton_attn.py", line 429, in forward ERROR 10-14 15:30:30 [core.py:588] unified_attention( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py", line 611, in unified_attention ERROR 10-14 15:30:30 [core.py:588] kernel_unified_attention_2d[( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/triton/runtime/jit.py", line 340, in ambda ERROR 10-14 15:30:30 [core.py:588] return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/triton/runtime/jit.py", line 681, in run ERROR 10-14 15:30:30 [core.py:588] kernel = self.compile( ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/triton/compiler/compiler.py", line 301, in compile ERROR 10-14 15:30:30 [core.py:588] next_module = compile_ir(module, metadata) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/triton/backends/hcu/compiler.py", line 355, in lambda ERROR 10-14 15:30:30 [core.py:588] stages["llir"] = lambda src, metadata: self.make_llir(src, metadata, options) ERROR 10-14 15:30:30 [core.py:588] File "/usr/local/lib/python3.10/dist-packages/triton/backends/hcu/compiler.py", line 284, in make_llir ERROR 10-14 15:30:30 [core.py:588] pm.run(mod) ERROR 10-14 15:30:30 [core.py:588] RuntimeError: PassManager::run failed Process EngineCore_0: Traceback (most recent call last): File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 590, in run_engine_core raise e File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 579, in run_engine_core engine_core.run_busy_loop() File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 606, in run_busy_loop self._process_engine_step() File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 631, in _process_engine_step outputs, model_executed = self.step_fn() File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 235, in step model_output = self.execute_model(scheduler_output) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 221, in execute_model raise err File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core.py", line 212, in execute_model return self.model_executor.execute_model(scheduler_output) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/executor/abstract.py", line 87, in execute_model output = self.collective_rpc("execute_model", File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc answer = run_method(self.driver_worker, method, args, kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2910, in run_method return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_worker.py", line 308, in execute_model output = self.model_runner.execute_model(scheduler_output, File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 1382, in execute_model model_output = self.model( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 564, in forward hidden_states = self.model(input_ids, positions, intermediate_tensors, File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 173, in __call__ return self.forward(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 379, in forward hidden_states, residual = layer( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 268, in forward hidden_states = self.self_attn( File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 198, in forward attn_output = self.attn(q, k, v) File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ERROR 10-14 15:30:30 [async_llm.py:419] AsyncLLM output_handler failed. ERROR 10-14 15:30:30 [async_llm.py:419] Traceback (most recent call last): ERROR 10-14 15:30:30 [async_llm.py:419] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/async_llm.py", line 378, in output_handler ERROR 10-14 15:30:30 [async_llm.py:419] outputs = await engine_core.get_output_async() ERROR 10-14 15:30:30 [async_llm.py:419] File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/core_client.py", line 740, in get_output_async ERROR 10-14 15:30:30 [async_llm.py:419] raise self._format_exception(outputs) from None ERROR 10-14 15:30:30 [async_llm.py:419] vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause. File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 243, in forward torch.ops.vllm.unified_attention_with_output( File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1116, in __call__ return self._op(*args, **(kwargs or {})) File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 452, in unified_attention_with_output self.impl.forward(self, File "/usr/local/lib/python3.10/dist-packages/vllm/v1/attention/backends/triton_attn.py", line 429, in forward unified_attention( File "/usr/local/lib/python3.10/dist-packages/vllm/attention/ops/triton_unified_attention.py", line 611, in unified_attention kernel_unified_attention_2d[( File "/usr/local/lib/python3.10/dist-packages/triton/runtime/jit.py", line 340, in lambda return lambda *args, **kwargs: self.run(grid=grid, warmup=False, *args, **kwargs) File "/usr/local/lib/python3.10/dist-packages/triton/runtime/jit.py", line 681, in run kernel = self.compile( File "/usr/local/lib/python3.10/dist-packages/triton/compiler/compiler.py", line 301, in compile next_module = compile_ir(module, metadata) File "/usr/local/lib/python3.10/dist-packages/triton/backends/hcu/compiler.py", line 355, in lambda stages["llir"] = lambda src, metadata: self.make_llir(src, metadata, options) File "/usr/local/lib/python3.10/dist-packages/triton/backends/hcu/compiler.py", line 284, in make_llir pm.run(mod) RuntimeError: PassManager::run failed INFO 10-14 15:30:30 [async_llm.py:345] Request chatcmpl-5180ad069ec2445c9bf8e769cd5ccd07 failed (engine dead). INFO: 127.0.0.1:36728 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [799]