INFO: 172.17.0.1:60358 - "POST /v1/chat/completions HTTP/1.1" 200 OK
INFO 06-17 01:06:11 [engine.py:328] Added request chatcmpl-f7d06f0651c34415b45f1224efed9ef4.
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop.
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Traceback (most recent call last):
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/multiproc_worker_utils.py", line 232, in _run_worker_process
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = run_method(worker, method, args, kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/utils.py", line 2604, in run_method
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 113, in start_worker_execution_loop
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.execute_model(execute_model_req=None)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 470, in execute_model
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.model_runner.execute_model(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1795, in execute_model
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_or_intermediate_states = model_executable(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl

--block-size 64 已经设置为64了，麻烦帮忙看看原因
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 552, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.model(input_ids, positions, intermediate_tensors,
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 172, in __call__
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self.forward(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 367, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states, residual = layer(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 256, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.self_attn(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 190, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] attn_output = self.attn(q, k, v)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 222, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] torch.ops.vllm.unified_attention_with_output(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1061, in __call__
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self_._op(*args, **(kwargs or {}))
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 416, in unified_attention_with_output
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] self.impl.forward(self,
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 896, in forward
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output[:num_prefill_tokens] = self.fa_prefix_attn_func( # noqa
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 1187, in vllm_flash_attn_varlen_func
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.vllm_mha_varlen_fwd(
(VllmWorkerProcess pid=304) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] RuntimeError: Paged KV cache block size must be 64
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop.
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Traceback (most recent call last):
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/multiproc_worker_utils.py", line 232, in _run_worker_process
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = run_method(worker, method, args, kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/utils.py", line 2604, in run_method
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 113, in start_worker_execution_loop
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.execute_model(execute_model_req=None)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 470, in execute_model
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.model_runner.execute_model(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1795, in execute_model
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_or_intermediate_states = model_executable(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 552, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.model(input_ids, positions, intermediate_tensors,
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 172, in __call__
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self.forward(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 367, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states, residual = layer(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 256, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.self_attn(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 190, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] attn_output = self.attn(q, k, v)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 222, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] torch.ops.vllm.unified_attention_with_output(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1061, in __call__
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self_._op(*args, **(kwargs or {}))
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 416, in unified_attention_with_output
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] self.impl.forward(self,
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 896, in forward
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output[:num_prefill_tokens] = self.fa_prefix_attn_func( # noqa
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 1187, in vllm_flash_attn_varlen_func
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.vllm_mha_varlen_fwd(
(VllmWorkerProcess pid=305) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] RuntimeError: Paged KV cache block size must be 64
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Exception in worker VllmWorkerProcess while processing method start_worker_execution_loop.
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] Traceback (most recent call last):
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/multiproc_worker_utils.py", line 232, in _run_worker_process
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = run_method(worker, method, args, kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/utils.py", line 2604, in run_method
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 113, in start_worker_execution_loop
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.execute_model(execute_model_req=None)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 470, in execute_model
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output = self.model_runner.execute_model(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return func(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1795, in execute_model
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_or_intermediate_states = model_executable(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 552, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.model(input_ids, positions, intermediate_tensors,
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 172, in __call__
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self.forward(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 367, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states, residual = layer(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 256, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] hidden_states = self.self_attn(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 190, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] attn_output = self.attn(q, k, v)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 222, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] torch.ops.vllm.unified_attention_with_output(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1061, in __call__
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] return self_._op(*args, **(kwargs or {}))
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 416, in unified_attention_with_output
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] self.impl.forward(self,
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 896, in forward
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] output[:num_prefill_tokens] = self.fa_prefix_attn_func( # noqa
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 1187, in vllm_flash_attn_varlen_func
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.vllm_mha_varlen_fwd(
(VllmWorkerProcess pid=303) ERROR 06-17 01:06:11 [multiproc_worker_utils.py:238] RuntimeError: Paged KV cache block size must be 64
ERROR 06-17 01:06:11 [engine.py:166] RuntimeError('Paged KV cache block size must be 64')
ERROR 06-17 01:06:11 [engine.py:166] Traceback (most recent call last):
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 164, in start
ERROR 06-17 01:06:11 [engine.py:166] self.run_engine_loop()
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 239, in run_engine_loop
ERROR 06-17 01:06:11 [engine.py:166] request_outputs = self.engine_step()
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 265, in engine_step
ERROR 06-17 01:06:11 [engine.py:166] raise e
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 248, in engine_step
ERROR 06-17 01:06:11 [engine.py:166] return self.engine.step()
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 1448, in step
ERROR 06-17 01:06:11 [engine.py:166] outputs = self.model_executor.execute_model(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 299, in execute_model
ERROR 06-17 01:06:11 [engine.py:166] driver_outputs = self._driver_execute_model(execute_model_req)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/executor/mp_distributed_executor.py", line 144, in _driver_execute_model
ERROR 06-17 01:06:11 [engine.py:166] return self.driver_worker.execute_model(execute_model_req)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 470, in execute_model
ERROR 06-17 01:06:11 [engine.py:166] output = self.model_runner.execute_model(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 06-17 01:06:11 [engine.py:166] return func(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1795, in execute_model
ERROR 06-17 01:06:11 [engine.py:166] hidden_or_intermediate_states = model_executable(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
ERROR 06-17 01:06:11 [engine.py:166] return self._call_impl(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
ERROR 06-17 01:06:11 [engine.py:166] return forward_call(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 552, in forward
ERROR 06-17 01:06:11 [engine.py:166] hidden_states = self.model(input_ids, positions, intermediate_tensors,
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 172, in __call__
ERROR 06-17 01:06:11 [engine.py:166] return self.forward(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 367, in forward
ERROR 06-17 01:06:11 [engine.py:166] hidden_states, residual = layer(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
ERROR 06-17 01:06:11 [engine.py:166] return self._call_impl(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
ERROR 06-17 01:06:11 [engine.py:166] return forward_call(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 256, in forward
ERROR 06-17 01:06:11 [engine.py:166] hidden_states = self.self_attn(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
ERROR 06-17 01:06:11 [engine.py:166] return self._call_impl(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
ERROR 06-17 01:06:11 [engine.py:166] return forward_call(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/qwen2.py", line 190, in forward
ERROR 06-17 01:06:11 [engine.py:166] attn_output = self.attn(q, k, v)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
ERROR 06-17 01:06:11 [engine.py:166] return self._call_impl(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
ERROR 06-17 01:06:11 [engine.py:166] return forward_call(*args, **kwargs)
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 222, in forward
ERROR 06-17 01:06:11 [engine.py:166] torch.ops.vllm.unified_attention_with_output(
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 1061, in __call__
ERROR 06-17 01:06:11 [engine.py:166] return self_._op(*args, **(kwargs or {}))
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/layer.py", line 416, in unified_attention_with_output
ERROR 06-17 01:06:11 [engine.py:166] self.impl.forward(self,
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/vllm/attention/backends/rocm_flash_attn.py", line 896, in forward
ERROR 06-17 01:06:11 [engine.py:166] output[:num_prefill_tokens] = self.fa_prefix_attn_func( # noqa
ERROR 06-17 01:06:11 [engine.py:166] File "/usr/local/lib/python3.10/dist-packages/flash_attn/flash_attn_interface.py", line 1187, in vllm_flash_attn_varlen_func
ERROR 06-17 01:06:11 [engine.py:166] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.vllm_mha_varlen_fwd(
ERROR 06-17 01:06:11 [engine.py:166] RuntimeError: Paged KV cache block size must be 64
ERROR 06-17 01:06:11 [serving_chat.py:885] Error in chat completion stream generator.
ERROR 06-17 01:06:11 [serving_chat.py:885] Traceback (most recent call last):
ERROR 06-17 01:06:11 [serving_chat.py:885] File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/serving_chat.py", line 487, in chat_completion_stream_generator
ERROR 06-17 01:06:11 [serving_chat.py:885] async for res in result_generator:
ERROR 06-17 01:06:11 [serving_chat.py:885] File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/client.py", line 671, in _process_request
ERROR 06-17 01:06:11 [serving_chat.py:885] raise request_output
ERROR 06-17 01:06:11 [serving_chat.py:885] vllm.engine.multiprocessing.MQEngineDeadError: Engine loop is not running. Inspect the stacktrace to find the original error: RuntimeError('Paged KV cache block size must be 64').
INFO: 172.17.0.1:60370 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
INFO: Shutting down
ERROR 06-17 01:06:11 [multiproc_worker_utils.py:120] Worker VllmWorkerProcess pid 303 died, exit code: -15
INFO 06-17 01:06:11 [multiproc_worker_utils.py:124] Killing local vLLM worker processes
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [1]
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 317, in _bootstrap
util._exit_function()
File "/usr/lib/python3.10/multiprocessing/util.py", line 357, in _exit_function
p.join()
File "/usr/lib/python3.10/multiprocessing/process.py", line 149, in join
res = self._popen.wait(timeout)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
pid, sts = os.waitpid(self.pid, flag)
File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 444, in signal_handler
raise KeyboardInterrupt("MQLLMEngine terminated")
KeyboardInterrupt: MQLLMEngine terminated