gem5-users@gem5.org

The gem5 Users mailing list

View all threads

Re: gem5 SVE vectoe length

等价无穷小
Mon, May 29, 2023 9:30 AM

Hi, Hoa,
Thanks for your reply and addvice, I am so sorry about to reply so lately, becacuse I got injuried seriously in last month.
I check the kernel booting log, I can see something like you have shown in your reply.

[    0.001241] CPU features: detected: 32-bit EL0 Support[    0.001246] CPU features: detected: 32-bit EL1 Support
[    0.001252] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.001261] CPU features: detected: Generic authentication (architected algorithm)
[    0.001268] CPU features: detected: LSE atomic instructions
[    0.001274] CPU features: detected: Privileged Access Never
[    0.001281] CPU features: detected: Scalable Vector Extension
[    0.005974] SVE: maximum available vector length 256 bytes per vector
[    0.005980] SVE: default vector length 64 bytes per vector
[    0.010960] CPU: All CPU(s) started at EL2

In the log information, the vector max size is 2048 bits, and the default vector size is 512 bits. 

Then, I use the svcntb() in my test program to get the vector size, and the output is like this:

SVE is available. The length is 512 bits

So, it can be proved that in my program will use the default vector size. And I use some code to input the sve register value, it can hold 8 double values.
but I cannot use the prctl system call to change the vector size in my code,  I can compile the code correctly, but when I run in gem5, it will produce a error:

gem5.opt: build/ARM/cpu/simple/timing.cc:473: virtual gem5::Fault gem5::TimingSimpleCPU::initiateMemRead(gem5::Addr, unsigned int, gem5::Request::Flags, const std::vector<bool>&): Assertion `split_addr <= addr || split_addr - addr < block_size' failed.Program aborted at tick 10964907276000

I will continue do something to figure out the mistake.

In your addvice, the error /lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).  We need add the -static option in our compiling time, I notice you have solve the problem.
And installing the lib required mannual is very difficult, I do not do it ever since.

I also try to use the inline assemble to use the msr instruction in my code. But when I run it in gem5, it will produce an illegle instruction error, I trace the mistake, I found it is the msr instruction.

Today, I try to change the default vector size in the gem5 source code, but I cannot figure out how to set the ZCR_EL register default values!
Maybe this method is not usable in the gem5,  but I do not know other methods to do so.

Regards,
Zhang Meng

等价无穷小
zhangm20@foxmail.com

Original Email

Sender:"Hoa Nguyen"< hoanguyen@ucdavis.edu >;

Sent Time:2023/4/21 1:34

To:"The gem5 Users mailing list"< gem5-users@gem5.org >;

Cc recipient:"等价无穷小"< zhangm20@foxmail.com >;

Subject:Re: [gem5-users] Re: gem5 SVE vectoe length

Hi Zhang,

I wanted to add that you can check the Linux kernel booting log to see whether the kernel recognized the maximum vector length of SVE correctly. Note that even if the SVE vlen is correctly recognized by the kernel, it doesn't mean the whole vlen is available to userspace. Though, I think it's worth checking.
During the boot process, the kernel will output something like this,

[    0.000384] CPU features: detected: Privileged Access Never
[    0.000385] CPU features: detected: LSE atomic instructions
[    0.000387] CPU features: detected: User Access Override
[    0.000389] CPU features: detected: 32-bit EL0 Support
[    0.000390] CPU features: detected: Scalable Vector Extension
[    0.000392] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.000394] CPU features: detected: Generic authentication (architected algorithm)
[    0.000396] CPU features: detected: 32-bit EL1 Support
[    0.001579] SVE: maximum available vector length 16 bytes per vector
[    0.001580] SVE: default vector length 16 bytes per vector
[    0.002819] CPU: All CPU(s) started at EL2

The SVE vector length in this case is 16 bytes, or 512 bits.

> On figuring out the content of ZCR_EL during the guest binary runtime.

I think Giacomo's suggestion is a proper way to do it. However, if it's not working for you, then there are a few things that you can do in the meantime,

  1. You can run gem5 and make a breakpoint somewhere when the guest binary is called.
  2. You also can run gem5 up until the binary is called in the guest, then take a checkpoint and inspect the checkpoint file. The checkpoint file should contain the value of the registers, including ZCR_EL1, ZCR_EL2, and ZCR_EL3. The ZCR_EL3 should contain the SVE vlen at all exception levels.

> The prctl system call, I read the manual about the function. And I use the system call in my test code. I use the -march=armv8-a+sve option, and I also try -msve-vector-bits=1024 to compile the program. I can get a binary. But when I run it in the gem5, it will produce a output like: /lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).  The sve is the binary I compiled . Then I check the glibc version in my simulator and host, it's 2.27. I try to install it by the source code in the image, but I can not operate it correctly.

I assume that you are doing cross-compilation and transferring the compiled binary from the host to the guest. You can add the "-static" flag to the compilation command so that all necessary libraries are statically linked to the output binary.
I think flag -march=armv8-a+sve alone should be sufficient for the compiler to generate SVE instructions.

Regards,
Hoa Nguyen

On Thu, Apr 20, 2023 at 9:22 AM 等价无穷小 via gem5-users <gem5-users@gem5.org> wrote:

Hi Giacomo,
Thanks for your birlliant reply, giving me some insights!  But there are some problems.

For the ZCR_EL register, I try to read or write the value from the register. I try to use the inline assembly, I use msr instruction. Then use the g++ -march=armv8-a+sve option to compile the test program, the result is there is no register named ZCR_EL. Maybe I do some wrong operation, but I'm not sure.

The /proc/sys/abi/sve_default_vector_length file. I cannot find this file in the image  I use. And I try to get some advice by google. It seemed I did not get the proper answer.

The prctl system call, I read the manual about the function. And I use the system call in my test code. I use the -march=armv8-a+sve option, and I also try -msve-vector-bits=1024 to compile the program. I can get a binary. But when I run it in the gem5, it will produce a output like: /lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).  The sve is the binary I compiled . Then I check the glibc version in my simulator and host, it's 2.27. I try to install it by the source code in the image, but I can not operate it correctly.

I think the methods you provide are useful and insightful. Maybe I do not operate correctly.  Is there some your suggestions?

Best regards,
Zhang Meng

等价无穷小
zhangm20@foxmail.com

Original Email

Sender:"Giacomo Travaglini via gem5-users"< gem5-users@gem5.org >;

Sent Time:2023/4/20 17:30

To:"The gem5 Users mailing list"< gem5-users@gem5.org >;

Cc recipient:"等价无穷小"< zhangm20@foxmail.com >;"Giacomo Travaglini"< giacomo.travaglini@arm.com >;

Subject:[gem5-users] Re: gem5 SVE vectoe length

Hi Zhang,

That parameter configures the maximum (hardware constrained) vector length.

It is possible to choose a different (smaller) vector length for a process by configuring the following registers

ZCR_EL1 1

ZCR_EL2 2

ZCR_EL3 3

So I believe the problem is that while you are properly setting up the gem5 parameter, those registers have been configured for a smaller vector length.

What happens if you change the vector length from the guest?

In Linux you can either change /proc/sys/abi/sve_default_vector_length (requires root permission) or use the prctl syscall 4 (from your program) with the PR_SVE_SET_VL flag.

Let me know if this works...

Kind Regards

Giacomo

On 20/04/2023 09:22, 等价无穷小 via gem5-users wrote:

Dear gem5 Community, 

I hope this email finds you well. I am currently working on a project that involves using ARM SVE in gem5 FS mode, and I have encountered a problem that I would appreciate your help with.

While using the ARM SVE in gem5 FS mode, I added the command line parameter "--param 'system.sve_vl = 16'" to specify the vector length. However, when I tried to retrieve the system vector length using the SVE intrinsic svcntb(), the result I obtained  was a vector length of 256 bits. This is different from the expected result based on the parameter I set.

I am unsure if I have missed any important steps or if there is an issue with my configuration. I would be grateful if you could provide guidance on this matter or point me to any relevant resources or examples that could help me resolve the issue.

Thank you in advance for your assistance, and I look forward to your response.

Best regards,

Zhang Meng

等价无穷小

zhangm20@foxmail.com

_______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-leave@gem5.org  IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose,  or store or copy the information in any medium. Thank you.


gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org

Hi, Hoa, Thanks for your reply and addvice, I am so sorry about to reply so lately, becacuse I got injuried seriously in last month. I check the kernel booting log, I can see something like you have shown in your reply. [ &nbsp; &nbsp;0.001241] CPU features: detected: 32-bit EL0 Support[ &nbsp; &nbsp;0.001246] CPU features: detected: 32-bit EL1 Support [ &nbsp; &nbsp;0.001252] CPU features: detected: Data cache clean to the PoU not required for I/D coherence [ &nbsp; &nbsp;0.001261] CPU features: detected: Generic authentication (architected algorithm) [ &nbsp; &nbsp;0.001268] CPU features: detected: LSE atomic instructions [ &nbsp; &nbsp;0.001274] CPU features: detected: Privileged Access Never [ &nbsp; &nbsp;0.001281] CPU features: detected: Scalable Vector Extension [ &nbsp; &nbsp;0.005974] SVE: maximum available vector length 256 bytes per vector [ &nbsp; &nbsp;0.005980] SVE: default vector length 64 bytes per vector [ &nbsp; &nbsp;0.010960] CPU: All CPU(s) started at EL2 In the log information, the vector max size is 2048 bits, and the default vector size is 512 bits.&nbsp; Then, I use the svcntb() in my test program to get the vector size, and the output is like this: SVE is available. The length is 512 bits So, it can be proved that in my program will use the default vector size. And I use some code to input the sve register value, it can hold 8 double values. but I cannot use the prctl system call to change the vector size in my code,&nbsp; I can compile the code correctly, but when I run in gem5, it will produce a error: gem5.opt: build/ARM/cpu/simple/timing.cc:473: virtual gem5::Fault gem5::TimingSimpleCPU::initiateMemRead(gem5::Addr, unsigned int, gem5::Request::Flags, const std::vector<bool&gt;&amp;): Assertion `split_addr <= addr || split_addr - addr < block_size' failed.Program aborted at tick 10964907276000 I will continue do something to figure out the mistake. In your addvice, the error&nbsp;/lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).&nbsp; We need add the -static option in our compiling time, I notice you have solve the problem. And installing the lib required mannual is very difficult, I do not do it ever since. I also try to use the inline assemble to use the msr instruction in my code. But when I run it in gem5, it will produce an illegle instruction error, I trace the mistake, I found it is the msr instruction. Today, I try to change the default vector size in the gem5 source code, but I cannot figure out how to set the ZCR_EL register default values! Maybe this method is not usable in the gem5,&nbsp; but I do not know other methods to do so. Regards, Zhang Meng 等价无穷小 zhangm20@foxmail.com Original Email Sender:"Hoa Nguyen"< hoanguyen@ucdavis.edu &gt;; Sent Time:2023/4/21 1:34 To:"The gem5 Users mailing list"< gem5-users@gem5.org &gt;; Cc recipient:"等价无穷小"< zhangm20@foxmail.com &gt;; Subject:Re: [gem5-users] Re: gem5 SVE vectoe length Hi Zhang, I wanted to add that you can check the Linux kernel booting log to see whether the kernel recognized the maximum vector length of SVE correctly. Note that even if the SVE&nbsp;vlen is&nbsp;correctly&nbsp;recognized by the kernel, it doesn't mean the whole vlen is available to userspace. Though, I think it's worth&nbsp;checking. During the boot process, the kernel will output something like this, [ &nbsp; &nbsp;0.000384] CPU features: detected: Privileged Access Never [ &nbsp; &nbsp;0.000385] CPU features: detected: LSE atomic instructions [ &nbsp; &nbsp;0.000387] CPU features: detected: User Access Override [ &nbsp; &nbsp;0.000389] CPU features: detected: 32-bit EL0 Support [ &nbsp; &nbsp;0.000390] CPU features: detected: Scalable Vector Extension [ &nbsp; &nbsp;0.000392] CPU features: detected: Data cache clean to the PoU not required for I/D coherence [ &nbsp; &nbsp;0.000394] CPU features: detected: Generic authentication (architected algorithm) [ &nbsp; &nbsp;0.000396] CPU features: detected: 32-bit EL1 Support [ &nbsp; &nbsp;0.001579] SVE: maximum available vector length 16 bytes per vector [ &nbsp; &nbsp;0.001580] SVE: default vector length 16 bytes per vector [ &nbsp; &nbsp;0.002819] CPU: All CPU(s) started at EL2 The SVE vector length in this case is 16 bytes, or 512 bits. &gt; On figuring out the content of&nbsp;ZCR_EL during the guest binary runtime. I think Giacomo's suggestion is a proper way to do it. However, if it's not working for you, then there are a few things that you can do in the meantime, 1. You can run gem5 and make a breakpoint somewhere when the guest binary is called. 2. You also can run gem5 up until the binary is called in the guest, then take a checkpoint and inspect the checkpoint file. The checkpoint file should contain the value of the registers, including ZCR_EL1, ZCR_EL2, and ZCR_EL3. The ZCR_EL3 should contain the SVE vlen at all exception levels. &gt;&nbsp;The prctl system call, I read the manual about the function. And I use the system call in my test code. I use the -march=armv8-a+sve option, and I also try -msve-vector-bits=1024 to compile the program. I can get a binary. But when I run it in the gem5, it will produce a output like: /lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).&nbsp; The sve is the binary I compiled . Then I check the glibc version in my simulator and host, it's 2.27. I try to install it by the source code in the image, but I can not operate it correctly. I assume that you are doing cross-compilation and transferring the compiled binary from the host to the guest. You can add the "-static" flag to the compilation command so that all necessary libraries are statically linked to the output binary. I think flag `-march=armv8-a+sve` alone should be sufficient for the compiler to generate SVE instructions. Regards, Hoa Nguyen On Thu, Apr 20, 2023 at 9:22 AM 等价无穷小 via gem5-users <gem5-users@gem5.org&gt; wrote: Hi Giacomo, Thanks for your birlliant reply, giving me some insights!&nbsp; But there are some problems. For the ZCR_EL register, I try to read or write the value from the register. I try to use the inline assembly, I use msr instruction. Then use the g++ -march=armv8-a+sve option to compile the test program, the result is there is no register named ZCR_EL. Maybe I do some wrong operation, but I'm not sure. The /proc/sys/abi/sve_default_vector_length file. I cannot find this file in the image&nbsp; I use. And I try to get some advice by google. It seemed I did not get the proper answer. The prctl system call, I read the manual about the function. And I use the system call in my test code. I use the -march=armv8-a+sve option, and I also try -msve-vector-bits=1024 to compile the program. I can get a binary. But when I run it in the gem5, it will produce a output like: /lib/aarch64-linux-gnu/glibc.so.6:version 'GLIBC_2.34' not found (required by ./sve).&nbsp; The sve is the binary I compiled . Then I check the glibc version in my simulator and host, it's 2.27. I try to install it by the source code in the image, but I can not operate it correctly. I think the methods you provide are useful and insightful. Maybe I do not operate correctly.&nbsp; Is there some your suggestions? Best regards, Zhang Meng 等价无穷小 zhangm20@foxmail.com Original Email Sender:"Giacomo Travaglini via gem5-users"< gem5-users@gem5.org &gt;; Sent Time:2023/4/20 17:30 To:"The gem5 Users mailing list"< gem5-users@gem5.org &gt;; Cc recipient:"等价无穷小"< zhangm20@foxmail.com &gt;;"Giacomo Travaglini"< giacomo.travaglini@arm.com &gt;; Subject:[gem5-users] Re: gem5 SVE vectoe length Hi Zhang, That parameter configures the *maximum* (hardware constrained) vector length. It is possible to choose a different (smaller) vector length for a process by configuring the following registers ZCR_EL1 [1] ZCR_EL2 [2] ZCR_EL3 [3] So I believe the problem is that while you are properly setting up the gem5 parameter, those registers have been configured for a smaller vector length. What happens if you change the vector length from the guest? In Linux you can either change /proc/sys/abi/sve_default_vector_length (requires root permission) or use the prctl syscall [4] (from your program) with the PR_SVE_SET_VL flag. Let me know if this works... Kind Regards Giacomo [1]: https://developer.arm.com/documentation/ddi0601/2020-12/AArch64-Registers/ZCR-EL1--SVE-Control-Register--EL1-?lang=en [2]: https://developer.arm.com/documentation/ddi0601/2020-12/AArch64-Registers/ZCR-EL2--SVE-Control-Register--EL2-?lang=en [3]: https://developer.arm.com/documentation/ddi0601/2020-12/AArch64-Registers/ZCR-EL3--SVE-Control-Register--EL3-?lang=en [4]: https://man7.org/linux/man-pages/man2/prctl.2.html On 20/04/2023 09:22, 等价无穷小 via gem5-users wrote: Dear gem5 Community, I hope this email finds you well. I am currently working on a project that involves using ARM SVE in gem5 FS mode, and I have encountered a problem that I would appreciate your help with. While using the ARM SVE in gem5 FS mode, I added the command line parameter "--param 'system.sve_vl = 16'" to specify the vector length. However, when I tried to retrieve the system vector length using the SVE intrinsic svcntb(), the result I obtained was a vector length of 256 bits. This is different from the expected result based on the parameter I set. I am unsure if I have missed any important steps or if there is an issue with my configuration. I would be grateful if you could provide guidance on this matter or point me to any relevant resources or examples that could help me resolve the issue. Thank you in advance for your assistance, and I look forward to your response. Best regards, Zhang Meng 等价无穷小 zhangm20@foxmail.com _______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-leave@gem5.org IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. _______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-leave@gem5.org