Hello,
When I'm conducting VEGA_X86 simulation in gem5, I encountered the following issue. My gem5 version is v23.0.0.1, and I haven't made any modifications to the files. The "square" benchmark compiles successfully using the command "HCC_AMDGPU_TARGET=gfx900 make".  Am I missing the correct GPU simulation support? How should I proceed? Thank you very much.

Here's the execution script:

build/VEGA_X86/gem5.opt configs/example/gpufs/vega10_kvm.py
--disk-image image/kernel/x86-gpu-fs-20220512.img
--kernel image/vmlinux-5.4.0-105-generic
--gpu-mmio-trace benchmark/gem5-resources/src/gpu-fs/vega_mmio.log
--app benchmark/gem5-resources/src/gpu/square/bin/square

Below is the output log:

src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x1fc) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x8b) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x480) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48d) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48e) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48f) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x490) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x485) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x486) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x488) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48a) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48b) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48c) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x491) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0xc0010015) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x4b564d05) unsupported by gem5. Skipping.
src/dev/x86/pc.cc:117: warn: Don't know what interrupt to clear for console.
src/dev/amdgpu/amdgpu_vm.hh:254: warn: Accessing unsupported MMIO aperture! Assuming NBIO
src/dev/amdgpu/amdgpu_vm.hh:270: warn: Accessing unsupported frame apperture!
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x4a not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x28 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x4a not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x12 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x11 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0xa0 not supported.
Exiting @ tick 15482304666500 because m5_exit instruction encountered
src/cpu/kvm/base.cc:570: hack: Pretending totalOps is equivalent to totalInsts()

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
128+0 records in
128+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.0080005 s, 16.4 MB/s
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/sysimgblt.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/sysfillrect.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/syscopyarea.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/fb_sys_fops.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/i2c/algos/i2c-algo-bit.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/gpu/drm/drm.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/gpu/drm/drm_kms_helper.ko fbdev_emulation=0
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdkcl.ko
[ 6.088345] amdkcl: loading out-of-tree module taints kernel.
[ 6.100346] amdkcl: module verification failed: signature and/or required key missing - tainting kernel
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdttm.ko
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amd-sched.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/iommu/amd_iommu_v2.ko
[ 6.192351] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel jroedel@suse.de
[ 6.196352] AMD-Vi: AMD IOMMUv2 functionality not available on this system
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdgpu.ko ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
[ 6.548374] [drm] amdgpu kernel modesetting enabled.
[ 6.552374] [drm] amdgpu version: 5.9.25
[ 6.552374] [drm] OS DRM version: 5.4.0
[ 6.568375] amdgpu: CRAT table not found

...
[ 11.740694] ---[ end trace c64e45806e3fef4a ]---
[ 11.744694] RIP: 0010:cpa_flush+0x10a/0x160
[ 11.748694] Code: f5 72 92 0f ae f0 48 8b 45 e0 65 48 33 04 25 28 00 00 00 75 57 48 83 c4 10 5b 41 5c 41 5d 5d c3 66 66 90 66 90 e9 34 ff ff ff <66> 0f ae 38 48 01 d0 48 39 c3 77 f4 eb bd ba 01 00 00 00 4c 89 e6
[ 11.752694] RSP: 0018:ffffc90000327298 EFLAGS: 00010287
[ 11.756695] RAX: ffff8880ac000000 RBX: ffff8880ac001000 RCX: ffff8880b31fe000
[ 11.760695] RDX: 0000000000000040 RSI: ffff888000000000 RDI: 80000000ac000173
[ 11.764695] RBP: ffffc900003272c0 R08: ffff888000000b00 R09: 0000000000000000
[ 11.768695] R10: ffffc90000327250 R11: 0000000000200000 R12: ffffc900003272e8
[ 11.772696] R13: 0000000000000000 R14: 0000000000000010 R15: 0000000000000200
[ 11.776696] FS: 00007fcc7c467540(0000) GS:ffff8880bca00000(0000) knlGS:0000000000000000
[ 11.780696] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 11.784696] CR2: 00007ffcea27ce88 CR3: 00000000b6f0c000 CR4: 00000000000006f0
[ 11.788697] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 11.792697] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
./script.sh: line 9: 490 Segmentation fault modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
Running benchmark/gem5-resources/src/gpu/square/bin/square
error: 'hipErrorInvalidDevice'(101) at square.cpp:58
Connection closed by foreign host.

| |
LinS@163.com
|
|
17861509600@163.com
|

[AMD Official Use Only - General]

Hi,

Can you show the output you removed?  What is being printed right before the crash?

Thanks,
Matt

From: Matt Sinclair mattdsinclair.wisc@gmail.com
Sent: Sunday, July 23, 2023 10:37 AM
To: The gem5 Users mailing list gem5-users@gem5.org
Cc: LinS@163.com 17861509600@163.com; Poremba, Matthew Matthew.Poremba@amd.com
Subject: Re: [gem5-users] gem5 VEGA_X86 simulation with GPU support

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi Lin,

I don't see anything obviously wrong with your command, but this error seems to imply that something with the setup of the GPU device is wrong.  If you didn't change anything though, then probably there is something wrong with our GPUFS instructions.  Matt P (CC'd) knows the GPUFS code much better than me though, so hopefully he can help more here.

Thanks,
Matt S.

On Sun, Jul 23, 2023 at 7:47 AM LinS--- via gem5-users <gem5-users@gem5.orgmailto:gem5-users@gem5.org> wrote:
Hello,
When I'm conducting VEGA_X86 simulation in gem5, I encountered the following issue. My gem5 version is v23.0.0.1, and I haven't made any modifications to the files. The "square" benchmark compiles successfully using the command "HCC_AMDGPU_TARGET=gfx900 make".  Am I missing the correct GPU simulation support? How should I proceed? Thank you very much.

Here's the execution script:

build/VEGA_X86/gem5.opt configs/example/gpufs/vega10_kvm.py
--disk-image image/kernel/x86-gpu-fs-20220512.img
--kernel image/vmlinux-5.4.0-105-generic
--gpu-mmio-trace benchmark/gem5-resources/src/gpu-fs/vega_mmio.log
--app benchmark/gem5-resources/src/gpu/square/bin/square

Below is the output log:

src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x1fc) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x8b) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x480) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48d) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48e) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48f) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x490) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x485) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x486) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x488) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48a) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48b) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x48c) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x491) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0xc0010015) unsupported by gem5. Skipping.
src/arch/x86/kvm/x86_cpu.cc:1562: warn: kvm-x86: MSR (0x4b564d05) unsupported by gem5. Skipping.
src/dev/x86/pc.cc:117: warn: Don't know what interrupt to clear for console.
src/dev/amdgpu/amdgpu_vm.hh:254: warn: Accessing unsupported MMIO aperture! Assuming NBIO
src/dev/amdgpu/amdgpu_vm.hh:270: warn: Accessing unsupported frame apperture!
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x4a not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x28 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x69 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x4a not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x12 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0x11 not supported.
src/dev/amdgpu/pm4_packet_processor.cc:326: warn: PM4 packet opcode 0xa0 not supported.
Exiting @ tick 15482304666500 because m5_exit instruction encountered
src/cpu/kvm/base.cc:570: hack: Pretending totalOps is equivalent to totalInsts()

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
128+0 records in
128+0 records out
131072 bytes (131 kB, 128 KiB) copied, 0.0080005 s, 16.4 MB/s
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/sysimgblt.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/sysfillrect.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/syscopyarea.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/video/fbdev/core/fb_sys_fops.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/i2c/algos/i2c-algo-bit.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/gpu/drm/drm.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/gpu/drm/drm_kms_helper.ko fbdev_emulation=0
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdkcl.ko
[ 6.088345] amdkcl: loading out-of-tree module taints kernel.
[ 6.100346] amdkcl: module verification failed: signature and/or required key missing - tainting kernel
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdttm.ko
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amd-sched.ko
insmod /lib/modules/5.4.0-105-generic/kernel/drivers/iommu/amd_iommu_v2.ko
[ 6.192351] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.demailto:jroedel@suse.de>
[ 6.196352] AMD-Vi: AMD IOMMUv2 functionality not available on this system
insmod /lib/modules/5.4.0-105-generic/updates/dkms/amdgpu.ko ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
[ 6.548374] [drm] amdgpu kernel modesetting enabled.
[ 6.552374] [drm] amdgpu version: 5.9.25
[ 6.552374] [drm] OS DRM version: 5.4.0
[ 6.568375] amdgpu: CRAT table not found

...
[ 11.740694] ---[ end trace c64e45806e3fef4a ]---
[ 11.744694] RIP: 0010:cpa_flush+0x10a/0x160
[ 11.748694] Code: f5 72 92 0f ae f0 48 8b 45 e0 65 48 33 04 25 28 00 00 00 75 57 48 83 c4 10 5b 41 5c 41 5d 5d c3 66 66 90 66 90 e9 34 ff ff ff <66> 0f ae 38 48 01 d0 48 39 c3 77 f4 eb bd ba 01 00 00 00 4c 89 e6
[ 11.752694] RSP: 0018:ffffc90000327298 EFLAGS: 00010287
[ 11.756695] RAX: ffff8880ac000000 RBX: ffff8880ac001000 RCX: ffff8880b31fe000
[ 11.760695] RDX: 0000000000000040 RSI: ffff888000000000 RDI: 80000000ac000173
[ 11.764695] RBP: ffffc900003272c0 R08: ffff888000000b00 R09: 0000000000000000
[ 11.768695] R10: ffffc90000327250 R11: 0000000000200000 R12: ffffc900003272e8
[ 11.772696] R13: 0000000000000000 R14: 0000000000000010 R15: 0000000000000200
[ 11.776696] FS: 00007fcc7c467540(0000) GS:ffff8880bca00000(0000) knlGS:0000000000000000
[ 11.780696] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 11.784696] CR2: 00007ffcea27ce88 CR3: 00000000b6f0c000 CR4: 00000000000006f0
[ 11.788697] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 11.792697] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
./script.sh: line 9: 490 Segmentation fault modprobe -v amdgpu ip_block_mask=0xff ppfeaturemask=0 dpm=0 audio=0
Running benchmark/gem5-resources/src/gpu/square/bin/square
error: 'hipErrorInvalidDevice'(101) at square.cpp:58
Connection closed by foreign host.

[Image removed by sender.]
LinS@163.com
17861509600@163.com

gem5-users mailing list -- gem5-users@gem5.orgmailto:gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.orgmailto:gem5-users-leave@gem5.org