Hello, I am trying to run a 16 core RISCV full system simulation with the following command: /build/RISCV/gem5.opt configs/example/riscv/fs_linux.py --kernel riscv-bootloader-vmlinux-5.10 --disk-image riscv_disk.img --caches --mem-size=256MB --mem-type=DDR4_2400_8x8 --cpu-type=AtomicSimpleCPU -n 16 The kernel and the bootloader are from the gem5-resources for Ubuntu riscv. However the simulation hangs after the line Starting simulation... In another setup, I chose to build the kernel with NR_CPUS=64 and use that. It successfully boots and finishes the script but with the below error - In the system.platform.terminal I can see the kernel fails to bring up more than 8 CPUs: [ 0.001142] smp: Bringing up secondary CPUs ... [ 1.024519] CPU8: failed to come online [ 2.049026] CPU9: failed to come online [ 3.073533] CPU10: failed to come online [ 4.098040] CPU11: failed to come online [ 5.122547] CPU12: failed to come online [ 6.147054] CPU13: failed to come online [ 7.171561] CPU14: failed to come online [ 8.196068] CPU15: failed to come online [ 8.196086] smp: Brought up 1 node, 8 CPUs I believe the issue is with the PLIC implementation of RISCV. Please let me know your thoughts on this. Thanks Nitish

Hi Nitish, The number of CPUs might also be limited by the bootloader as well. You might want to recompile the bootloader using the following documentation https://github.com/gem5/gem5-resources/tree/stable/src/riscv-fs#bootloader-bbl . You might want to modify this parameter in the proxy kernel source code, https://github.com/riscv-software-src/riscv-pk/blob/710c23a5bbeecf171ac86d6e39d275af8f176354/machine/mtrap.h#L9, which limits the number of hardware threads. Note that MAX_HARTS must follow this constraint, https://github.com/riscv-software-src/riscv-pk/blob/710c23a5bbeecf171ac86d6e39d275af8f176354/machine/mtrap.c#L104 . Regards, Hoa Nguyen On Fri, Nov 17, 2023, 04:24 Nitish Arya via gem5-users <gem5-users@gem5.org> wrote: > > Hello, > > I am trying to run a 16 core RISCV full system simulation with the > following command: > > /build/RISCV/gem5.opt configs/example/riscv/fs_linux.py --kernel > riscv-bootloader-vmlinux-5.10 --disk-image riscv_disk.img --caches > --mem-size=256MB --mem-type=DDR4_2400_8x8 --cpu-type=AtomicSimpleCPU -n 16 > > The kernel and the bootloader are from the gem5-resources for Ubuntu riscv. > > However the simulation hangs after the line Starting simulation... > > In another setup, I chose to build the kernel with NR_CPUS=64 and use > that. It successfully boots and finishes the script but with the below > error - > > In the system.platform.terminal I can see the kernel fails to bring up > more than 8 CPUs: > > [ 0.001142] smp: Bringing up secondary CPUs ... > [ 1.024519] CPU8: failed to come online > [ 2.049026] CPU9: failed to come online > [ 3.073533] CPU10: failed to come online > [ 4.098040] CPU11: failed to come online > [ 5.122547] CPU12: failed to come online > [ 6.147054] CPU13: failed to come online > [ 7.171561] CPU14: failed to come online > [ 8.196068] CPU15: failed to come online > [ 8.196086] smp: Brought up 1 node, 8 CPUs > > > I believe the issue is with the PLIC implementation of RISCV. > > Please let me know your thoughts on this. > > Thanks > > Nitish > > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-leave@gem5.org >

Ah, also, if modifying MAX_HARTS doesn't work then you can follow the following PRs to use a newer bootloader, which works with 16 cores. - Building the bootloader https://github.com/gem5/gem5-resources/pull/13 - Changing gem5 to use the new bootloader https://github.com/gem5/gem5/pull/390#issuecomment-1750191706. Regards, Hoa Nguyen On Fri, Nov 17, 2023 at 11:22 PM Hoa Nguyen <hoanguyen@ucdavis.edu> wrote: > Hi Nitish, > > The number of CPUs might also be limited by the bootloader as well. You > might want to recompile the bootloader using the following documentation > https://github.com/gem5/gem5-resources/tree/stable/src/riscv-fs#bootloader-bbl > . > > You might want to modify this parameter in the proxy kernel source code, > https://github.com/riscv-software-src/riscv-pk/blob/710c23a5bbeecf171ac86d6e39d275af8f176354/machine/mtrap.h#L9, > which limits the number of hardware threads. > Note that MAX_HARTS must follow this constraint, > https://github.com/riscv-software-src/riscv-pk/blob/710c23a5bbeecf171ac86d6e39d275af8f176354/machine/mtrap.c#L104 > . > > Regards, > Hoa Nguyen > > On Fri, Nov 17, 2023, 04:24 Nitish Arya via gem5-users < > gem5-users@gem5.org> wrote: > >> >> Hello, >> >> I am trying to run a 16 core RISCV full system simulation with the >> following command: >> >> /build/RISCV/gem5.opt configs/example/riscv/fs_linux.py --kernel >> riscv-bootloader-vmlinux-5.10 --disk-image riscv_disk.img --caches >> --mem-size=256MB --mem-type=DDR4_2400_8x8 --cpu-type=AtomicSimpleCPU -n 16 >> >> The kernel and the bootloader are from the gem5-resources for Ubuntu >> riscv. >> >> However the simulation hangs after the line Starting simulation... >> >> In another setup, I chose to build the kernel with NR_CPUS=64 and use >> that. It successfully boots and finishes the script but with the below >> error - >> >> In the system.platform.terminal I can see the kernel fails to bring up >> more than 8 CPUs: >> >> [ 0.001142] smp: Bringing up secondary CPUs ... >> [ 1.024519] CPU8: failed to come online >> [ 2.049026] CPU9: failed to come online >> [ 3.073533] CPU10: failed to come online >> [ 4.098040] CPU11: failed to come online >> [ 5.122547] CPU12: failed to come online >> [ 6.147054] CPU13: failed to come online >> [ 7.171561] CPU14: failed to come online >> [ 8.196068] CPU15: failed to come online >> [ 8.196086] smp: Brought up 1 node, 8 CPUs >> >> >> I believe the issue is with the PLIC implementation of RISCV. >> >> Please let me know your thoughts on this. >> >> Thanks >> >> Nitish >> >> _______________________________________________ >> gem5-users mailing list -- gem5-users@gem5.org >> To unsubscribe send an email to gem5-users-leave@gem5.org >> >