Hello, I would like to confirm whether the following observation is correct: Is the gem5 RISC-V simulation of riscv-ubuntu-20.04-boot approximately 30 times slower than the equivalent workload for x86 and aarch64? Or could there be an error in my setup? Scenario: I am booting a full-system simulation on all three platforms (RISC-V, x86, and ARM) using their respective gem5 resource workloads for Ubuntu 20.04. I'm utilizing an ATOMIC processor and 4GB of memory for these tests. To measure performance, I am timing how long it takes to reach the m5 exit point. I have used the same custom script for all platforms. While the x86 and ARM platforms take less than 4 minutes to boot, the RISC-V simulation takes around 110 minutes. I also tried running the sample configuration riscv-ubuntu-run.py, but changed the processor to ATOMIC. The boot time remained equally lengthy. For comparison, the riscv-fs.py script (which uses riscv-disk-img, a different image) boots in under 4 minutes. My testing environment is an x86 i9-13900K system with 64GB of memory, running Ubuntu 22.04.3 LTS. I've tested with build/ALL/gem5.fast and build/RISC-V/gem5.fast, both built from gem5 stable version 23.0.1.0. Is the RISC-V simulation inherently slower, or is the Ubuntu image for RISC-V more resource-intensive than those for ARM and x86? Best regards, Daniel

Hi Daniel, You should see some minute differences in the total boot time for each of the ISAs. However here are some reasons why you're seeing this kind of difference in the boot times. I am assuming that you're using the default pre-built disk image resources from Gem5 Resources. The reason that you are seeing such a high difference between the RISCV and the ARM disk images for Ubuntu 20.04 is because `/sbin/init` is different for both of these disk images. In the RISCV disk image, the default behavior is to boot Ubuntu with systemd, which takes a lot of time. Typically booting that disk image using ATOMIC CPU takes about 2 hours in my go-to config that I use. In the ARM disk image however, the `/sbin/init` is modified to just mount /proc, /sys and the block device and `m5 exit` is then executed. This takes only a couple of minutes as it skips systemd. The changes are described here https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/arm-ubuntu/disk-image/arm-ubuntu/arm-ubuntu.json There is an init_old in the same disk image which I guess you can trigger by passing init=/sbin/init_old (I have not tested). I'd suggest a different experiment to study the boot times. You should either pass init=/bin/bash or build a ramdisk of the same kernel for each of the ISAs with /sbin/m5 as the only other program on the disk and then measure the boot time. You can follow these instructions to build a RISCV ramdisk: https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/riscv-boot-exit-nodisk/ I have not worked on the X86 disk image directly, but you can find information on that disk image and how that was built here https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/x86-ubuntu/ So I don't know why the X86 disk image is booting so fast. Just ensure that you're not using KVM. Also make sure you're booting x86 Ubuntu with systemd via the terminal output. Thanks, Kaustav. On Wed, Aug 30, 2023 at 4:19 AM Daniel Sperry via gem5-users < gem5-users@gem5.org> wrote: > Hello, > > I would like to confirm whether the following observation is correct: Is > the gem5 RISC-V simulation of riscv-ubuntu-20.04-boot approximately 30 > times slower than the equivalent workload for x86 and aarch64? Or could > there be an error in my setup? > > Scenario: I am booting a full-system simulation on all three platforms > (RISC-V, x86, and ARM) using their respective gem5 resource workloads for > Ubuntu 20.04. I'm utilizing an ATOMIC processor and 4GB of memory for these > tests. To measure performance, I am timing how long it takes to reach the > m5 exit point. I have used the same custom script for all platforms. While > the x86 and ARM platforms take less than 4 minutes to boot, the RISC-V > simulation takes around 110 minutes. > > I also tried running the sample configuration riscv-ubuntu-run.py, but > changed the processor to ATOMIC. The boot time remained equally lengthy. > > For comparison, the riscv-fs.py script (which uses riscv-disk-img, a > different image) boots in under 4 minutes. > > My testing environment is an x86 i9-13900K system with 64GB of memory, > running Ubuntu 22.04.3 LTS. I've tested with build/ALL/gem5.fast and > build/RISC-V/gem5.fast, both built from gem5 stable version 23.0.1.0. > > Is the RISC-V simulation inherently slower, or is the Ubuntu image for > RISC-V more resource-intensive than those for ARM and x86? > > Best regards, > Daniel > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-leave@gem5.org >

Hi Kaustav, Thank you for your in-depth explanation; it was immensely helpful. I added 'init=/root/gem5_init.sh' to the kernel args and reduced RISC-V ATOMIC boot time to under 2 minutes. Achieving a fast boot time with Ubuntu on RISC-V was my primary objective, and I'm glad it worked. Unfortunately, the available documentation did not offer clear guidance on how to achieve this. For those interested, here's the code snippet: workload = Workload("riscv-ubuntu-20.04-boot") kernel_args = board.get_default_kernel_args() kernel_args.append("init=/root/gem5_init.sh") workload.set_parameter("kernel_args", kernel_args) On another note, I tried to modify the image using QEMU to mimic the ARM image setup, but it resulted in an unusable QEMU image (which is expected according to https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/arm-ubuntu#manipulating-the-disk-image ). I haven't tested this modified image with gem5, but I presume it would have worked. However, using 'init' gives me more control anyway. Concerning the x86 simulation, my initial timing was off. The boot time with an ATOMIC processor and the default Ubuntu workload (with systemd) takes about 16 minutes on my machine. Here is the configuration: memory = DualChannelDDR4_2400(size="3GB") processor = SimpleProcessor(cpu_type=CPUTypes.ATOMIC, num_cores=2, isa=ISA.X86) workload = Workload("x86-ubuntu-18.04-boot") x86 simulation time: 16m42.347s Terminal output snippet: systemd[1]: Detected architecture x86-64. Welcome to Ubuntu 18.04.2 LTS! Once again, thank you for the detailed answer! Best regards, Daniel On Wed, 30 Aug 2023 at 18:36, Kaustav Goswami via gem5-users < gem5-users@gem5.org> wrote: > Hi Daniel, > > You should see some minute differences in the total boot time for each of > the ISAs. However here are some reasons why you're seeing this kind of > difference in the boot times. > > I am assuming that you're using the default pre-built disk image resources > from Gem5 Resources. The reason that you are seeing such a high difference > between the RISCV and the ARM disk images for Ubuntu 20.04 is because > `/sbin/init` is different for both of these disk images. In the RISCV disk > image, the default behavior is to boot Ubuntu with systemd, which takes a > lot of time. Typically booting that disk image using ATOMIC CPU takes about > 2 hours in my go-to config that I use. In the ARM disk image however, the > `/sbin/init` is modified to just mount /proc, /sys and the block device and > `m5 exit` is then executed. This takes only a couple of minutes as it skips > systemd. The changes are described here > https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/arm-ubuntu/disk-image/arm-ubuntu/arm-ubuntu.json > There is an init_old in the same disk image which I guess you can trigger > by passing init=/sbin/init_old (I have not tested). I'd suggest a different > experiment to study the boot times. You should either pass init=/bin/bash > or build a ramdisk of the same kernel for each of the ISAs with /sbin/m5 as > the only other program on the disk and then measure the boot time. You can > follow these instructions to build a RISCV ramdisk: > https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/riscv-boot-exit-nodisk/ > > I have not worked on the X86 disk image directly, but you can find > information on that disk image and how that was built here > https://gem5.googlesource.com/public/gem5-resources/+/refs/heads/stable/src/x86-ubuntu/ So > I don't know why the X86 disk image is booting so fast. Just ensure that > you're not using KVM. Also make sure you're booting x86 Ubuntu with systemd > via the terminal output. > > Thanks, > Kaustav. > > On Wed, Aug 30, 2023 at 4:19 AM Daniel Sperry via gem5-users < > gem5-users@gem5.org> wrote: > >> Hello, >> >> I would like to confirm whether the following observation is correct: Is >> the gem5 RISC-V simulation of riscv-ubuntu-20.04-boot approximately 30 >> times slower than the equivalent workload for x86 and aarch64? Or could >> there be an error in my setup? >> >> Scenario: I am booting a full-system simulation on all three platforms >> (RISC-V, x86, and ARM) using their respective gem5 resource workloads for >> Ubuntu 20.04. I'm utilizing an ATOMIC processor and 4GB of memory for these >> tests. To measure performance, I am timing how long it takes to reach the >> m5 exit point. I have used the same custom script for all platforms. While >> the x86 and ARM platforms take less than 4 minutes to boot, the RISC-V >> simulation takes around 110 minutes. >> >> I also tried running the sample configuration riscv-ubuntu-run.py, but >> changed the processor to ATOMIC. The boot time remained equally lengthy. >> >> For comparison, the riscv-fs.py script (which uses riscv-disk-img, a >> different image) boots in under 4 minutes. >> >> My testing environment is an x86 i9-13900K system with 64GB of memory, >> running Ubuntu 22.04.3 LTS. I've tested with build/ALL/gem5.fast and >> build/RISC-V/gem5.fast, both built from gem5 stable version 23.0.1.0. >> >> Is the RISC-V simulation inherently slower, or is the Ubuntu image for >> RISC-V more resource-intensive than those for ARM and x86? >> >> Best regards, >> Daniel >> _______________________________________________ >> gem5-users mailing list -- gem5-users@gem5.org >> To unsubscribe send an email to gem5-users-leave@gem5.org >> > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-leave@gem5.org > -- Daniel Sperry