#### 宿主机信息： GPU：8张K100_AI 内存：940G，查看free内存还有540多G #### vllm 容器内 /var/log/hylog 输出日志 ```shell [ROCT-Thunk-Interface_5.7_docker/src/libhsakmt.c:40, ERROR] iocti fail, ret: -1, erro: 12 [ROCT-Thunk-Interface_5.7_docker/src/memory.c:206, ERROR] [hsakmtAllocMemory] failed to allocate 1503657984 bytes from device [ROCT-Thunk-Interface_5.7_docker/src/libhsakmt.c:40, ERROR] iocti fail, ret: -1, Arno: 12 [ROCT-Thunk-Interface_5.7_docker/src/memory.c:206, ERROR] [hsakutAllockenory] failed to allocate 1503657984 bytes from device [ROCR-Runtime_5.7_docker/src/core/runtime/hsa_ext_and.cpp:148, ERROR] HSA exception: MemoryRegion::BlockAllocator::alloc failed. [ROCT-Thunk-Interface 5.7_docker/src/libhsakmt.c:40, EMR］ ioctl fail, ret： -1, ermno: 12 [ROCT-Thunk-Interface 5.7_docker/src/memory.c:206, ERROR] [hsakmtAlloMenory] failed to allocate 1925185536 bytes from device [ROCT-Thunk-Interface_5.7_docker/src/libhsakmt.c:40, ERROR] iocti fail, ret: -1, errno: 12 [ROCT-Thunk-Interface_5.7_docker/src/memory.c:206, ERROR] [hsakmtAllocMemory] failed to allocate 1925185536 bytes from device [ROCR-Runtime_5.7_docker/src/core/runtime/hsa_ext_amd.cpp:148, ERROR] HSA exception: MemoryRegion::BlockAllocator::alloc failed. ``` #### VLLM推理容器启动命令 ```shell port=6215 log_path=/data/llm/logs model_path=/data/llm/TT3-32B cuda_devices=0,1,2,3 gpu_memory_utilization=0.9 max_model_len=65536 max_num_seqs=32 max_num_batched_tokens=131072 timestamp=`date -u -d +8hours +"%Y%m%d"` model_name=$(basename "$model_path") num_gpus=$(echo $cuda_devices | tr ',' '\n' | wc -l) log_dir=$log_path/$port.$timestamp.[$model_name] mkdir -p $log_dir docker run -d --privileged --restart=always --shm-size=32G --group-add video --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --network=host --ipc=host \ --name $port.$timestamp.$model_name \ -v $model_path:/models \ -v $log_dir:/logs \ -v /data/hylog:/var/log/hylog \ -v /opt/hyhal:/opt/hyhal:ro \ -v /sys/kernel/debug:/sys/kernel/debug:ro \ -e TZ=Asia/Shanghai \ -e "HIP_VISIBLE_DEVICES=$cuda_devices" \ -e "VLLM_ENGINE_ITERATION_TIMEOUT_S=300" \ -e "VLLM_LOGGING_LEVEL=DEBUG" \ -e "CUDA_LAUNCH_BLOCKING=1" \ -e "NCCL_DEBUG=TRACE" \ -e "GLOO_SOCKET_IFNAME=docker0" \ --device=/dev/kfd \ --device=/dev/dri \ --device=/dev/mkfd \ image.sourcefind.cn:5000/dcu/admin/base/vllm:0.9.2-ubuntu22.04-dtk25.04.2-py3.10 \ bash -lc "vllm serve /models --trust-remote-code --host 0.0.0.0 --port $port --served-model-name $model_name --tokenizer-mode auto \ --tensor-parallel-size $num_gpus --gpu-memory-utilization $gpu_memory_utilization --max-model-len $max_model_len --max-num-seqs $max_num_seqs --max-num-batched-tokens $max_num_batched_tokens \ --dtype float16 2>&1 | tee -a /logs/server.log" ```