3
41
1
29
运行时
提问帖
有回复
新人贴
服务器8张卡,型号为K100,部署Qwen3-VL-2B-Instruct模型提示识别不到config.json中的vocab_size
2026-01-12 23:22:46
98

wen3_vl_inference  | ERROR 01-12 23:20:43 [engine.py:476] AttributeError: 'Qwen3VLConfig' object has no attribute 'vocab_size'

qwen3_vl_inference  | Traceback (most recent call last):

qwen3_vl_inference  |   File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap

qwen3_vl_inference  |     self.run()

qwen3_vl_inference  |   File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run

qwen3_vl_inference  |     self._target(*self._args, **self._kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 478, in run_mp_engine

qwen3_vl_inference  |     raise e from None

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 464, in run_mp_engine

qwen3_vl_inference  |     engine = MQLLMEngine.from_vllm_config(

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 139, in from_vllm_config

qwen3_vl_inference  |     return cls(

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/multiprocessing/engine.py", line 93, in __init__

qwen3_vl_inference  |     self.engine = LLMEngine(*args, **kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 273, in __init__

qwen3_vl_inference  |     self.model_executor = executor_class(vllm_config=vllm_config)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 53, in __init__

qwen3_vl_inference  |     self._init_executor()

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 48, in _init_executor

qwen3_vl_inference  |     self.collective_rpc("load_model")

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/uniproc_executor.py", line 57, in collective_rpc

qwen3_vl_inference  |     answer = run_method(self.driver_worker, method, args, kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/utils/__init__.py", line 2910, in run_method

qwen3_vl_inference  |     return func(*args, **kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 211, in load_model

qwen3_vl_inference  |     self.model_runner.load_model()

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/worker/model_runner.py", line 1189, in load_model

qwen3_vl_inference  |     self.model = get_model(vllm_config=self.vllm_config)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/__init__.py", line 59, in get_model

qwen3_vl_inference  |     return loader.load_model(vllm_config=vllm_config,

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/base_loader.py", line 38, in load_model

qwen3_vl_inference  |     model = initialize_model(vllm_config=vllm_config,

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/model_loader/utils.py", line 67, in initialize_model

qwen3_vl_inference  |     return model_class(vllm_config=vllm_config, prefix=prefix)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/compilation/decorators.py", line 154, in __init__

qwen3_vl_inference  |     old_init(self, vllm_config=vllm_config, prefix=prefix, **kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/transformers.py", line 443, in __init__

qwen3_vl_inference  |     self.model = TransformersModel(vllm_config=vllm_config, prefix=prefix)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/transformers.py", line 195, in __init__

qwen3_vl_inference  |     config.vocab_size,

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/transformers/configuration_utils.py", line 207, in __getattribute__

qwen3_vl_inference  |     return super().__getattribute__(key)

qwen3_vl_inference  | AttributeError: 'Qwen3VLConfig' object has no attribute 'vocab_size'

qwen3_vl_inference  | WARNING: Logging before InitGoogleLogging() is written to STDERR

qwen3_vl_inference  | W0112 23:20:44.749639   477 ProcessGroupNCCL.cpp:1279] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())

qwen3_vl_inference  | Traceback (most recent call last):

qwen3_vl_inference  |   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main

qwen3_vl_inference  |     return _run_code(code, main_globals, None,

qwen3_vl_inference  |   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code

qwen3_vl_inference  |     exec(code, run_globals)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1495, in <module>

qwen3_vl_inference  |     uvloop.run(run_server(args))

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 69, in run

qwen3_vl_inference  |     return loop.run_until_complete(wrapper())

qwen3_vl_inference  |   File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/uvloop/__init__.py", line 48, in wrapper

qwen3_vl_inference  |     return await main

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1431, in run_server

qwen3_vl_inference  |     await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1451, in run_server_worker

qwen3_vl_inference  |     async with build_async_engine_client(args, client_config) as engine_client:

qwen3_vl_inference  |   File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__

qwen3_vl_inference  |     return await anext(self.gen)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 158, in build_async_engine_client

qwen3_vl_inference  |     async with build_async_engine_client_from_engine_args(

qwen3_vl_inference  |   File "/usr/lib/python3.10/contextlib.py", line 199, in __aenter__

qwen3_vl_inference  |     return await anext(self.gen)

qwen3_vl_inference  |   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 291, in build_async_engine_client_from_engine_args

qwen3_vl_inference  |     raise RuntimeError(

qwen3_vl_inference  | RuntimeError: Engine process failed to start. See stack trace for the root cause.



这是详细的错误信息,观察发现应该是镜像中还是用的Qwen2的那一套读取,是格式不支持吗?


我的docker-compose内容如下:

version: '3.8'


services:

 qwen3-vl:

   # 切换为官方推荐的标准镜像

   image: image.sourcefind.cn:5000/dcu/admin/base/vllm:0.9.2-ubuntu22.04-dtk25.04.2-py3.10

   container_name: qwen3_vl_inference

   privileged: true

   network_mode: "host" # 官方推荐,降低网络延迟

   shm_size: '200gb'    # 官方推荐,DCU 必填大数值

   user: root

   devices:

     - /dev/kfd:/dev/kfd

     - /dev/dri:/dev/dri

     - /dev/mkfd:/dev/mkfd

   group_add:

     - video

   cap_add:

     - SYS_PTRACE

   security_opt:

     - seccomp:unconfined

   volumes:

     - /home/zxy_projects/all-models/Qwen/Qwen3-VL-2B-Instruct:/model/Qwen3-VL-2B-Instruct

     - /opt/hyhal:/opt/hyhal:ro

   environment:

     - HIP_VISIBLE_DEVICES=5 # 使用完全空闲的 5 号卡

     - VLLM_USE_V1=0         # 关键:Qwen3 目前在 vLLM V0 引擎下更稳定

     - LD_LIBRARY_PATH=/opt/dtk/lib:/opt/rocm/lib:/usr/local/lib:/opt/hyhal/lib:$LD_LIBRARY_PATH

   command: >

     bash -c "pip install av -i https://mirrors.aliyun.com/pypi/simple/ &&

     python3 -m vllm.entrypoints.openai.api_server

     --model /model/Qwen3-VL-2B-Instruct

     --trust-remote-code

     --gpu-memory-utilization 0.8

     --max-model-len 4096

     --port 40001"

   restart: always

全部评论(1)
只看楼主
M
minfy
1楼
Qwen3-VL模型从vllm0.11.0开始支持,请从光源下载vllm0.11.0镜像进行测试,我这边验证服务可以正常启动。
光源地址:https://developer.sourcefind.cn/servicelist/detail?post_id=61036870-b3c7-11f0-9989-acde48001122&active=TagDownload
1/15
没有更多内容了
--
楼主
Level.6
0 帖子
楼主热帖
暂无数据