我在K100-AI上部署的gptq量化模型没法正常运行,用vllm:0.9.2-ubuntu22.04-dtk25.04.1-rc5-rocblas104381-0915-das1.6-py3.10-20250916-rc2镜像模型会一直输出感叹号,用vllm:0.8.5-rockylinux8.6-dtk25.04.1-rc3-das1.6-py3.10-20250724-shca镜像则加载过程中报错。报错内容如下:
torch._dynamo.exc.Unsupported: Graph break due to unsupported builtin lmslimquant.PyCapsule.gptq_gemm. This function is either a Python builtin (e.g. _warnings.warn) or a third-party C/C++ Python extension (perhaps created with pybind). If it is a Python builtin, please file an issue on GitHub so the PyTorch team can add support for it and see the next case for a workaround. If it is a third-party C/C++ Python extension, please either wrap it into a PyTorch-understood custom operator (see https://pytorch.org/tutorials/advanced/custom_ops_landing_page.html for more details) or, if it is traceable, use torch.compiler.allow_in_graph.