我使用四节点。

环境变量配置：

export ALLREDUCE_STREAM_WITH_COMPUTE=1

export NCCL_SOCKET_IFNAME=ib0

export GLOO_SOCKET_IFNAME=ib0

export NCCL_IB_HCA=ib0 # 环境中的IB网卡名字

unset NCCL_ALGO

export NCCL_MIN_NCHANNELS=16

export NCCL_MAX_NCHANNELS=16

export NCCL_NET_GDR_READ=1

export HIP_VISIBLE_DEVICES=0,1,2,3,4,5,6,7

export VLLM_SPEC_DECODE_EAGER=1

export VLLM_MLA_DISABLE=0

export VLLM_USE_FLASH_MLA=1

# 海光CPU绑定核

export VLLM_NUMA_BIND=1

export VLLM_RANK0_NUMA=0

export VLLM_RANK1_NUMA=1

export VLLM_RANK2_NUMA=2

export VLLM_RANK3_NUMA=3

export VLLM_RANK4_NUMA=4

export VLLM_RANK5_NUMA=5

export VLLM_RANK6_NUMA=6

export VLLM_RANK7_NUMA=7

#每个计算节点分别配置

export VLLM_HOST_IP=12.7.8.47

export VLLM_HOST_IP=12.7.8.48

export VLLM_HOST_IP=12.7.8.49

export VLLM_HOST_IP=12.7.8.50

vllm命令：

vllm serve /public/ai_data/models/DeepSeek-V3.2-BF16  \

--served-model-name ds32 \

--trust-remote-code \

--chat-template ./custom_template.jinja \

--distributed-executor-backend ray \

--dtype bfloat16 \

--enforce-eager \

--tensor-parallel-size 32 \

--enable-expert-parallel \

--max-model-len 4096 \

--max-num-batched-tokens 4096 \

--max-num-seqs 32 \

--disable-log-requests \

--block-size 64 \

--no-enable-chunked-prefill \

--no-enable-prefix-caching \

--gpu-memory-utilization 0.9

custom_template.jinja的内容：

{% for message in messages %}{% if message['role'] == 'user' %}<|im_start|>user

{{ message['content'] }}<|im_end|>

{% elif message['role'] == 'assistant' %}<|im_start|>assistant

{{ message['content'] }}<|im_end|>

{% endif %}{% endfor %}<|im_start|>assistant

curl测试：

curl http://localhost:8000/v1/chat/completions \

  -H "Content-Type: application/json" \

  -d '{

    "model": "ds32",

    "messages": [{"role": "user", "content": "讲个笑话"}],

    "max_tokens": 400,

    "stream": false

  }'

输出是乱码：

íííííííííííííííííígramí Â冊握\"即三隨- 之 íí之之之之í之之 respectively是自\r途各有í分別

我的问题是：我哪里搞错了，这是模型权重问题还是vllm的适配问题。

有没有详细的拉起满血deepseek的教程。

我想运行多dp，ep的场景。镜像版本是不是支持。