我有一台服务器是8张64G的K100-AI,运行Qwen3.5-122B-A10B模型,具体部署如下:
docker容器启动:
qwen3.5:
image: harbor.sourcefind.cn:5443/dcu/admin/base/custom:vllm018-ubuntu22.04-dtk26.04-gemma4-0413
container_name: qwen3.5
privileged: true
user: root
network_mode: host
ipc: host
shm_size: 1024G
devices:
- "/dev/kfd:/dev/kfd"
- "/dev/mkfd:/dev/mkfd"
- "/dev/dri:/dev/dri"
volumes:
- "/etc/localtime:/etc/localtime:ro"
- "/opt/hyhal:/opt/hyhal"
- "/data/deploy/ds70/ds70:/data/ds70"
- "/data/deploy/models:/data/models"
group_add:
- video
environment:
- LANG=zh_CN.UTF-8
- LANGUAGE=zh_CN.UTF-8
cap_add:
- SYS_PTRACE
security_opt:
- seccomp=unconfined
ulimits:
memlock:
soft: -1
hard: -1
command: /bin/bash
tty: true
stdin_open: true
restart: unless-stopped
vllm启动:
nohup vllm serve /data/models/Qwen3.5-122B-A10B
--port 16663
--tensor-parallel-size 8
--reasoning-parser qwen3
--enable-auto-tool-choice
--tool-call-parser qwen3_coder
--disable-custom-all-reduce
--gpu-memory-utilization 0.925
--default-chat-template-kwargs '{"enable_thinking": true}'
--served-model-name Qwen3.5-122B-A10B &
然后通过ragflow接入大模型,在进行问答过程中发现没有返回思考过程返回到前端。
通过以下脚本直接调用,可以看到think过程
from openai import OpenAI
client = OpenAI(
base_url="http://192.168.0.103:16663/v1",
api_key="anything" # vLLM 不需要真实 key
)
response = client.chat.completions.create(
model="Qwen3.5-122B-A10B",
messages=[{"role": "user", "content": "你的问题"}],
temperature=1.0,
max_tokens=8192
)
print("=== 思考过程 ===")
print(response.choices[0].message.reasoning or "无思考过程")
print("\n=== 最终答案 ===")
print(response.choices[0].message.content)