机器配置使用的 rock-5.7.1-6.2.26-V1.5.aio.run 、DTK-24.04-Kylinv10SP2-x86_64.tar.gz、image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.5.0-dtk24.04.1-ubuntu20.04-py310-20250225

运行的指令是

docker run -d --name vllm-qwen2.5-14b \

--network=host \

--ipc=host \

--shm-size=16G \

--device=/dev/kfd \

--device=/dev/mkfd \

--device=/dev/dri \

--userns=host \

--restart=always \

-v /opt/hyhal:/opt/hyhal \

-v /data/models:/data/models \

--group-add video  \

--privileged \

--security-opt seccomp=unconfined \

-e HIP_LAUNCH_BLOCKING=1 \

-e TORCH_USE_HIP_DSA=1 \

-e HSA_FORCE_FINE_GRAIN_MAPPING=1 \

image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.5.0-dtk24.04.1-ubuntu20.04-py310-20250225 \

python -m vllm.entrypoints.openai.api_server \

--model /data/models/Qwen2___5-14B-Instruct \

--served-model-name qwen2.5-14b-instruct \

--disable-sliding-window \

--api-key vllm \

--tensor-parallel-size 1 \

--max-model-len 8192  \

--gpu-memory-utilization 0.85 \

--enforce-eager \

--trust-remote-code \

--port 8000

______  ____________  ____  ________  __

  / __ \ \/ /_  __/ __ \/ __ \/ ____/ / / /

 / /_/ /\  / / / / / / / /_/ / /   / /_/ /

/ ____/ / / / / / /_/ / _, _/ /___/ __  /  

/_/     /_/ /_/  \____/_/ |_|\____/_/ /_/  

DTK version 24.04.1

pytorch version 2.1.0

1 DCU were detected in the container.

NOTE: Make sure the user is in the 39 group.

For more information, please go to https://developer.hpccube.com/

INFO 03-31 22:18:50 api_server.py:177] vLLM API server version 0.5.0

INFO 03-31 22:18:50 api_server.py:178] args: Namespace(host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key='vllm', lora_modules=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], model='/data/models/Qwen2___5-14B-Instruct', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, download_dir=None, load_format='auto', dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=8192, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=1, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=False, disable_sliding_window=True, use_v2_block_manager=False, num_lookahead_slots=0, seed=0, swap_space=4, gpu_memory_utilization=0.85, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=True, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, device='auto', image_input_type=None, image_token_id=None, image_input_shape=None, image_feature_size=None, image_processor=None, image_processor_revision=None, disable_image_processor=False, scheduler_delay_factor=0.0, enable_chunked_prefill=False, speculative_model=None, num_speculative_tokens=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, model_loader_extra_config=None, preemption_mode=None, served_model_name=['qwen2.5-14b-instruct'], qlora_adapter_name_or_path=None, engine_use_ray=False, disable_log_requests=False, max_log_len=None)

INFO 03-31 22:18:50 llm_engine.py:161] Initializing an LLM engine (v0.5.0) with config: model='/data/models/Qwen2___5-14B-Instruct', speculative_config=None, tokenizer='/data/models/Qwen2___5-14B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=8192, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), seed=0, served_model_name=qwen2.5-14b-instruct)

INFO 03-31 22:18:51 selector.py:55] Using ROCmFlashAttention backend.

WARNING: Logging before InitGoogleLogging() is written to STDERR

I0331 22:18:51.389006     1 ProcessGroupNCCL.cpp:686] [Rank 0] ProcessGroupNCCL initialization options:NCCL_ASYNC_ERROR_HANDLING: 1, NCCL_DESYNC_DEBUG: 0, NCCL_ENABLE_TIMING: 0, NCCL_BLOCKING_WAIT: 0, TIMEOUT(ms): 1800000, USE_HIGH_PRIORITY_STREAM: 0, TORCH_DISTRIBUTED_DEBUG: OFF, NCCL_DEBUG: OFF, ID=94027216494688

I0331 22:18:52.565218     1 ProcessGroupNCCL.cpp:1340] NCCL_DEBUG: N/A

I0331 22:18:52.567260     1 ProcessGroupNCCL.cpp:686] [Rank 0] ProcessGroupNCCL initialization options:NCCL_ASYNC_ERROR_HANDLING: 1, NCCL_DESYNC_DEBUG: 0, NCCL_ENABLE_TIMING: 0, NCCL_BLOCKING_WAIT: 0, TIMEOUT(ms): 1800000, USE_HIGH_PRIORITY_STREAM: 0, TORCH_DISTRIBUTED_DEBUG: OFF, NCCL_DEBUG: OFF, ID=94027229983984

I0331 22:18:52.573663     1 ProcessGroupNCCL.cpp:686] [Rank 0] ProcessGroupNCCL initialization options:NCCL_ASYNC_ERROR_HANDLING: 1, NCCL_DESYNC_DEBUG: 0, NCCL_ENABLE_TIMING: 0, NCCL_BLOCKING_WAIT: 0, TIMEOUT(ms): 1800000, USE_HIGH_PRIORITY_STREAM: 0, TORCH_DISTRIBUTED_DEBUG: OFF, NCCL_DEBUG: OFF, ID=94027223676528

WARNING 03-31 22:18:52 __init__.py:104] Model architecture Qwen2ForCausalLM is partially supported by ROCm: Sliding window attention is not yet supported in ROCm's flash attention

Traceback (most recent call last):

 File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main

   return _run_code(code, main_globals, None,

 File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code

   exec(code, run_globals)

 File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 196, in <module>

   engine = AsyncLLMEngine.from_engine_args(

 File "/usr/local/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 395, in from_engine_args

   engine = cls(

 File "/usr/local/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 349, in __init__

   self.engine = self._init_engine(*args, **kwargs)

 File "/usr/local/lib/python3.10/site-packages/vllm/engine/async_llm_engine.py", line 470, in _init_engine

   return engine_class(*args, **kwargs)

 File "/usr/local/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 223, in __init__

   self.model_executor = executor_class(

 File "/usr/local/lib/python3.10/site-packages/vllm/executor/executor_base.py", line 41, in __init__

   self._init_executor()

 File "/usr/local/lib/python3.10/site-packages/vllm/executor/gpu_executor.py", line 24, in _init_executor

   self.driver_worker.load_model()

 File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 121, in load_model

   self.model_runner.load_model()

 File "/usr/local/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 147, in load_model

   self.model = get_model(

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/model_loader/__init__.py", line 21, in get_model

   return loader.load_model(model_config=model_config,

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 249, in load_model

   model = _initialize_model(model_config, self.load_config,

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/model_loader/loader.py", line 100, in _initialize_model

   return model_class(config=model_config.hf_config,

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 323, in __init__

   self.model = Qwen2Model(config, cache_config, quant_config)

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 249, in __init__

   self.layers = nn.ModuleList([

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 250, in <listcomp>

   Qwen2DecoderLayer(config, cache_config, quant_config)

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 183, in __init__

   self.self_attn = Qwen2Attention(

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 134, in __init__

   self.rotary_emb = get_rope(

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 705, in get_rope

   rotary_emb = RotaryEmbedding(head_size, rotary_dim, max_position, base,

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 66, in __init__

   cache = self._compute_cos_sin_cache()

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 87, in _compute_cos_sin_cache

   inv_freq = self._compute_inv_freq(self.base)

 File "/usr/local/lib/python3.10/site-packages/vllm/model_executor/layers/rotary_embedding.py", line 81, in _compute_inv_freq

   inv_freq = 1.0 / (base**(torch.arange(

 File "/usr/local/lib/python3.10/site-packages/torch/_tensor.py", line 39, in wrapped

   return handle_torch_function(wrapped, args, *args, **kwargs)

 File "/usr/local/lib/python3.10/site-packages/torch/overrides.py", line 1560, in handle_torch_function

   result = mode.__torch_function__(public_api, types, args, kwargs)

 File "/usr/local/lib/python3.10/site-packages/torch/utils/_device.py", line 77, in __torch_function__

   return func(*args, **kwargs)

 File "/usr/local/lib/python3.10/site-packages/torch/_tensor.py", line 40, in wrapped

   return f(*args, **kwargs)

 File "/usr/local/lib/python3.10/site-packages/torch/_tensor.py", line 938, in __rpow__

   return torch.pow(other, self)

RuntimeError: HIP error: the operation cannot be performed in the present state

Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.

I0331 22:18:54.867456     1 ProcessGroupNCCL.cpp:874] [Rank 0] Destroyed 1communicators on CUDA device 0

这个是需要更新vllm吗