在k100-AI环境下，使用vllm0.6.6启动deepseek推理服务时
1、单机单卡启动deepseek服务正常，也能正常进行推理问答
2、单机多卡启动deepseek服务时报错，错误信息如下:

INFO 04-30 18:43:37 utils.py:918] Found nccl from library librccl.so.1
(VllmWorkerProcess pid=626) INFO 04-30 18:43:37 utils.py:918] Found nccl from library librccl.so.1
(VllmWorkerProcess pid=628) INFO 04-30 18:43:37 utils.py:918] Found nccl from library librccl.so.1
(VllmWorkerProcess pid=627) INFO 04-30 18:43:37 utils.py:918] Found nccl from library librccl.so.1
INFO 04-30 18:43:37 pynccl.py:69] vLLM is using nccl==2.18.3
(VllmWorkerProcess pid=628) INFO 04-30 18:43:37 pynccl.py:69] vLLM is using nccl==2.18.3
(VllmWorkerProcess pid=627) INFO 04-30 18:43:37 pynccl.py:69] vLLM is using nccl==2.18.3
(VllmWorkerProcess pid=626) INFO 04-30 18:43:37 pynccl.py:69] vLLM is using nccl==2.18.3
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] Exception in worker VllmWorkerProcess while processing method init_device.
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] Traceback (most recent call last):
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 230, in _run_worker_process
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 148, in init_device
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     init_worker_distributed_environment(self.vllm_config, self.rank,
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 470, in init_worker_distributed_environment
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     ensure_model_parallel_initialized(parallel_config.tensor_parallel_size,
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 1101, in ensure_model_parallel_initialized
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     initialize_model_parallel(tensor_model_parallel_size,
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 1045, in initialize_model_parallel
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     _TP = init_model_parallel_group(group_ranks,
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 876, in init_model_parallel_group
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     return GroupCoordinator(
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 216, in __init__
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.pynccl_comm = PyNcclCommunicator(
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl.py", line 99, in __init__
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.comm: ncclComm_t = self.nccl.ncclCommInitRank(
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl_wrapper.py", line 275, in ncclCommInitRank
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.NCCL_CHECK(self._funcs["ncclCommInitRank"](ctypes.byref(comm),
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl_wrapper.py", line 254, in NCCL_CHECK
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     raise RuntimeError(f"NCCL error: {error_str}")
(VllmWorkerProcess pid=628) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] RuntimeError: NCCL error: unhandled system error (run with NCCL_DEBUG=INFO for details)
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] Exception in worker VllmWorkerProcess while processing method init_device.
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] Traceback (most recent call last):
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 230, in _run_worker_process
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 148, in init_device
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     init_worker_distributed_environment(self.vllm_config, self.rank,
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/worker/worker.py", line 470, in init_worker_distributed_environment
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     ensure_model_parallel_initialized(parallel_config.tensor_parallel_size,
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 1101, in ensure_model_parallel_initialized
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     initialize_model_parallel(tensor_model_parallel_size,
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 1045, in initialize_model_parallel
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     _TP = init_model_parallel_group(group_ranks,
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 876, in init_model_parallel_group
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     return GroupCoordinator(
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/parallel_state.py", line 216, in __init__
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.pynccl_comm = PyNcclCommunicator(
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl.py", line 99, in __init__
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.comm: ncclComm_t = self.nccl.ncclCommInitRank(
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl_wrapper.py", line 275, in ncclCommInitRank
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     self.NCCL_CHECK(self._funcs["ncclCommInitRank"](ctypes.byref(comm),
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]   File "/usr/local/lib/python3.10/site-packages/vllm/distributed/device_communicators/pynccl_wrapper.py", line 254, in NCCL_CHECK
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236]     raise RuntimeError(f"NCCL error: {error_str}")
(VllmWorkerProcess pid=627) ERROR 04-30 18:43:37 multiproc_worker_utils.py:236] RuntimeError: NCCL error: unhandled system error (run with NCCL_DEBUG=INFO for details)
ERRORTask exception was never retrieved
future: <Task finished name='Task-2' coro=<MQLLMEngineClient.run_output_handler_loop() done, defined at /usr/local/lib/python3.10/site-packages/vllm/engine/multiprocessing/client.py:178> exception=ZMQError('Operation not supported')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/multiprocessing/client.py", line 184, in run_output_handler_loop
    while await self.output_socket.poll(timeout=VLLM_RPC_TIMEOUT
  File "/usr/local/lib/python3.10/site-packages/zmq/_future.py", line 372, in poll
    raise _zmq.ZMQError(_zmq.ENOTSUP)
zmq.error.ZMQError: Operation not supported
ERRORTask exception was never retrieved
future: <Task finished name='Task-3' coro=<MQLLMEngineClient.run_output_handler_loop() done, defined at /usr/local/lib/python3.10/site-packages/vllm/engine/multiprocessing/client.py:178> exception=ZMQError('Operation not supported')>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/vllm/engine/multiprocessing/client.py", line 184, in run_output_handler_loop
    while await self.output_socket.poll(timeout=VLLM_RPC_TIMEOUT
  File "/usr/local/lib/python3.10/site-packages/zmq/_future.py", line 372, in poll
    raise _zmq.ZMQError(_zmq.ENOTSUP)
zmq.error.ZMQError: Operation not supported
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 774, in <module>
    uvloop.run(run_server(args))
  File "/usr/local/lib/python3.10/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
  File "/usr/local/lib/python3.10/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 740, in run_server
    async with build_async_engine_client(args) as engine_client:
  File "/usr/local/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 118, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/usr/local/lib/python3.10/contextlib.py", line 199, in __aenter__
    return await anext(self.gen)
  File "/usr/local/lib/python3.10/site-packages/vllm/entrypoints/openai/api_server.py", line 223, in build_async_engine_client_from_engine_args
    raise RuntimeError(
RuntimeError: Engine process failed to start. See stack trace for the root cause.
+ echo 'Error: Command failed with exit code 1'
Error: Command failed with exit code 1
+ exit 1
root@ai-80588aeb196f45a899065038de12d2c2-deployment-84cb7d779c-2m2jc:~# /usr/local/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 12 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
command terminated with exit code 137

====================================

关于本问题，容器镜像替换为vllm0.7.2对应镜像后，问题fix

vllm072对应容器镜像：image.sourcefind.cn:5000/dcu/admin/base/custom:vllm0.7.2-ubuntu22.04-dtk25.04-rc7-das1.5-py3.10-20250510-fixpy