2
基础环境
DTK 24.04.2部署MM_GroundingDINO报错
2025-05-27 18:31:48
143

基础环境:
系统:ky10
DTK: 24.04.2
Torch: 2.1.0 + 1.16.0
Python: 3.10
MMCV: 2.0.1

具体报错: #基础环境#  

Traceback (most recent call last):
  File "/root/ovd/ovd_mmdet/./tools/train.py", line 121, in <module>
    main()
  File "/root/ovd/ovd_mmdet/./tools/train.py", line 110, in main
    runner = Runner.from_cfg(cfg)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/runner/runner.py", line 462, in from_cfg
    runner = cls(
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/runner/runner.py", line 429, in __init__
    self.model = self.build_model(model)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/runner/runner.py", line 836, in build_model
    model = MODELS.build(model)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/root/ovd/ovd_mmdet/mmdet/models/detectors/ovd_dino.py", line 69, in __init__
    super().__init__(*args, **kwargs)
  File "/root/ovd/ovd_mmdet/mmdet/models/detectors/deformable_detr.py", line 69, in __init__
    super().__init__(*args, decoder=decoder, bbox_head=bbox_head, **kwargs)
  File "/root/ovd/ovd_mmdet/mmdet/models/detectors/base_detr.py", line 77, in __init__
    self._init_layers()
  File "/root/ovd/ovd_mmdet/mmdet/models/detectors/ovd_dino.py", line 163, in _init_layers
    self.language_model = MODELS.build(self.language_model_cfg)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
    return self.build_func(cfg, *args, **kwargs, registry=self)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
    return build_from_cfg(cfg, registry, default_args)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
    obj = obj_cls(**args)  # type: ignore
  File "/root/ovd/ovd_mmdet/mmdet/models/language_models/bert.py", line 123, in __init__
    BertEncoder(
  File "/root/ovd/ovd_mmdet/mmdet/models/language_models/bert.py", line 205, in __init__
    self.model = HFBertModel.from_pretrained(
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 309, in _wrapper
    return func(*args, **kwargs)
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4252, in from_pretrained
    device_in_context = get_torch_context_manager_or_global_device()
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 322, in get_torch_context_manager_or_global_device
    default_device = torch.get_default_device()
  File "/data1/zdhs/miniconda3/envs/ovd_mmdet_t210_py310/lib/python3.10/site-packages/torch/__init__.py", line 1841, in __getattr__
    raise AttributeError(f"module '{__name__}' has no attribute '{name}'")
AttributeError: module 'torch' has no attribute 'get_default_device'

相关配置具体如下:

uname -a
Linux localhost.localdomain 4.19.90-89.23.v2401.ky10.x86_64 #1 SMP Fri Apr 25 12:25:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

dtk:
lrwxrwxrwx  1 root root   16 May 27 16:39 dtk -> /opt/dtk-24.04.2
drwxr-xr-x 37 root root 4096 Sep 20  2024 dtk-24.04.2

pytorch:
torch-2.1.0+das.opt1.dtk24042-cp310-cp310-manylinux_2_28_x86_64.whl
torchvision-0.16.0+das.opt1.dtk24042-cp310-cp310-manylinux_2_28_x86_64.whl

mmcv:
mmcv-2.0.1+das.opt1.dtk24042-cp310-cp310-manylinux_2_28_x86_64.whl

rocm架构:
rocminfo | grep gfx
Name:                    gfx928   
Name:                    amdgcn-amd-amdhsa--gfx928:sramecc+:xnack-

pytorch信息:
>>> import torch
>>> print(torch.__version__)
2.1.0
>>> print(torch.version.hip)
5.7.24311
>>> print(torch.cuda.is_available())
True
>>> print(torch.cuda.get_device_name(0))
K100_AI
>>> print(torch.cuda.get_arch_list())
['gfx906', 'gfx926', 'gfx928']
>>> print(torch.cuda.get_device_capability())
(9, 2)
全部评论(7)
只看楼主
R
renzhc
2楼
单独执行import torch; torch.get_default_device()也会报这个错误吗
2025/5/28
网
网友hmmhiH
回复 @renzhc
不会报错,但是执行b= torch.zeros((4,4),dtype=torch.float32, device="cuda")报错
2025/5/28
R
renzhc
回复 @renzhc
这个的报错信息可以贴详细一点吗
2025/5/28
网
网友hmmhiH
回复 @renzhc
当我将DTK升级到25.04并且更新驱动时,这个问题貌似解决了
2025/5/28
全部5条回复
网
网友hmmhiH
楼主
1楼

测试demo报错:

import torch

print("Device:".torch.cuda.get device name(0))

export AMD SERIALIZE KERNEL=3

export HIP LAUNCH BLOCKING=1

python - <<EOF

b= torch.zeros((4,4),dtype=torch.float32, device="cuda")

torch.cuda.synchronize()

EOF

Device:K100 AITraceback(most recent call last):

File"<stdin>",line 6,in <module>

RuntimeError: HIP error:invalid device function

Compile with`TORCH USE HIP DSA` to enable device-side assertions.
2025/5/28
没有更多内容了
--
楼主
Level.2
0 帖子
楼主热帖
暂无数据