1
1
2
DTK开发
vllm 测试版本,报错 RuntimeError: FlashAttention forward only supports head dimension at most 128
2024-08-05 16:50:37
126
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]     return super().apply(*args, **kwargs)  # type: ignore[misc]
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]   File "/usr/local/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 254, in forward
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _flash_attn_varlen_forward(
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]   File "/usr/local/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 52, in _flash_attn_varlen_forward
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.varlen_fwd(
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] RuntimeError: FlashAttention forward only supports head dimension at most 128
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] 
(VllmWorkerProcess pid=92341) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] Exception in worker VllmWorkerProcess while processing method determine_num_available_blocks: FlashAttention forward only supports head dimension at most 128, Traceback (most recent call last):
全部评论(1)
只看楼主
P
panhuiwen
1楼
设计如此,目前fa还不支持head dim>128的情况,我们后面会逐步完善fa上的功能,请关注das发布公告
2024/8/12
没有更多内容了
--
楼主
Level.1
0 帖子
楼主热帖
暂无数据