3
模型应用
Ktransformers v0.3.1 部署 Qwen3-235B-BF16 报错
2025-09-17 10:05:52
176
本机环境: CPU:Intel(R) Xeon(R) Silver 4410Y DCU:K100_AI 镜像:image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.4.1-ubuntu22.04-dtk25.04-py3.10 Ktransformers:v0.3.1 启动命令:python3 ktransformers/server/main.py --model_path /mnt/model/Qwen3-235B-A22B-Instruct-2507-GGUF/BF16/ --gguf_path /mnt/model/Qwen3-235B-A22B-Instruct-2507-GGUF/BF16/ --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml --cpu_infer 36 报错信息: no balance_serve flashinfer not found, use triton for linux Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1034, in from_pretrained config_class = CONFIG_MAPPING[config_dict["model_type"]] File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 736, in __getitem__ raise KeyError(key) KeyError: 'qwen3_moe' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/root/ktransformers/ktransformers/server/main.py", line 122, in model main() File "/root/ktransformers/ktransformers/server/main.py", line 108, in main args = arg_parser.parse_args() File "/usr/local/lib/python3.10/site-packages/ktransformers/server/args.py", line 139, in parse_args model_config = AutoConfig.from_pretrained(args.model_dir, trust_remote_code=True) File "/usr/local/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1036, in from_pretrained raise ValueError( ValueError: The checkpoint you are trying to load has model type Qwen3-Moe but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date. 另外部署 DeepSeek-R1-BF16 启动需要大半天才行,有什么能够优化。
全部评论(8)
只看楼主
L
litzh
1楼
出现 KeyError: 'qwen3_moe', 应该是 transformers<4.51.0导致的。可以使用pip install 更新 transformers到>=4.51.0
2025/9/17
网
网友yfri9U
回复 @litzh
升级完成之后,报新的错误,但是部署 DeepSeek-R1 是可以的,看代码理论上 DeepSeek 在这块也会报错啊,是有什么特殊处理吗 python3 ktransformers/server/main.py --model_path /mnt/model/Qwen3-235B-A22B-Instruct-2507-GGUF/BF16/ --gguf_path /mnt/model/Qwen3-235B-A22B-Instruct-2507-GGUF/BF16/ --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml --cpu_infer 36 no balance_serve flashinfer not found, use triton for linux [2025-09-17 15:12:04,135] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect) Traceback (most recent call last): File "/root/ktransformers/ktransformers/server/main.py", line 122, in main() File "/root/ktransformers/ktransformers/server/main.py", line 109, in main create_interface(config=cfg, default_args=cfg) File "/usr/local/lib/python3.10/site-packages/ktransformers/server/utils/create_interface.py", line 30, in create_interface GlobalInterface.interface = BackendInterface(default_args) File "/usr/local/lib/python3.10/site-packages/ktransformers/server/backend/interfaces/ktransformers.py", line 49, in __init__ self.model = custom_models[config.architectures[0]](config) KeyError: 'Qwen3MoeForCausalLM'
2025/9/17
东
东风
回复 @网友yfri9U
你这个报错我遇到过,也很好改,改好后依然报其他错。KT只支持使用balance_serve运行Qwen3模型,DCU无法运行Qwen3,原因是balance_serve使用FlashInfer进行推理,FlashInfer强制调用cuda_fp8.h头文件,这个头文件在dtk的根目录中中并不存在。
2025/9/17
网
网友yfri9U
回复 @东风
无解了吧
2025/9/17
全部7条回复
没有更多内容了
--
楼主
Level.1
0 帖子
楼主热帖
暂无数据