容器镜像:image.sourcefind.cn:5000/dcu/admin/base/vllm:0.8.5-ubuntu22.04-dtk25.04.1-rc5-das1.6-py3.10-20250724
容器创建:docker run -it --shm-size=500G --network host -v /mnt/:/home/ -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name ds {image} bash
启动:vllm serve /home/DeepSeek-R1-Distill-Llama-70B/ --tensor-parallel-size 8 --max-model-len 32768 --dtype float16 --served-model-name dpsk --gpu-memory-utilization 0.85 --enforce-eager --port 8000
提问:curl -H "Authorization: Bearer hyxino123" http://192.168.110.192:8000/v1/completions -H "Content-Type: application/json" -d '{
"model": "dpsk",
"prompt": "甲乙两班共有学生98人,甲班比乙班多6人,求两班各有多少人?",
"max_tokens": 2048,
"temperature": 0
}'
回复如下:
回复乱码问题有相应的解决方法吗?急用!!!