当前vllm暂未进行GLM4的适配,请参考glm4-9b_pytorch目录下的README进行使用。
basic_demo目录下:python openai_api_server.py 启动,然后调用/v1/chat/completions接口发生报错,一堆/data/jenkins_workspace/workspace/vllm@3/csrc/attention/dtype_bfloat16_hip.cuh:93: float2 vllm::bf1622float2(const __nv_bfloat162): Device-side assertion `false' failed.

暂无数据