设计如此,目前fa还不支持head dim>128的情况,我们后面会逐步完善fa上的功能,请关注das发布公告
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] return super().apply(*args, **kwargs) # type: ignore[misc]
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] File "/usr/local/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 254, in forward
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _flash_attn_varlen_forward(
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] File "/usr/local/lib/python3.10/site-packages/flash_attn/flash_attn_interface.py", line 52, in _flash_attn_varlen_forward
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = flash_attn_cuda.varlen_fwd(
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] RuntimeError: FlashAttention forward only supports head dimension at most 128
(VllmWorkerProcess pid=92338) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226]
(VllmWorkerProcess pid=92341) ERROR 08-05 16:37:17 multiproc_worker_utils.py:226] Exception in worker VllmWorkerProcess while processing method determine_num_available_blocks: FlashAttention forward only supports head dimension at most 128, Traceback (most recent call last):

暂无数据