* Using the pre-built docker images (gcr.io/gem5-test/ubuntu*) I can

* When trying to build natively on apple silicon, (python3 and python

Dear all, I am trying to build gem5 in an (arm) mac machine and I am encountering some problems. If anyone has managed to do so, I would be grateful if they could share some pointers. Here are the various methods that I have tried so far: * Using the pre-built docker images (gcr.io/gem5-test/ubuntu*) I can successfully build and run gem5, but since the images are build for x86, recompilation takes really long time. * When trying to build natively on apple silicon, (python3 and python development headers/python3-config installed from brew), scons complains about not finding a working python installation. If could someone write where the log files of scons are, I might be able to dig deeper. I have tried also re-building the docker images on mac (using the dockerfiles from repo) because this way the underline ubuntu image is an ARM one and emulation of x86 on docker won't be needed. Unfortunately: * On ubuntu 20 with gcc-10 and ubuntu 20 with all dependencies images, scons again reports not finding a working python installation * On ubuntu 22 with all dependencies, looks like there is an incompatibility among protobuf version and the version required by the headers, so compilation throws an error. * On ubuntu 20 with minimum dependencies image, gem builds successfully but throws an error on runtime since it tries to access the (dynamically linked? ) qemu-x86_64 (qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory). Could this be recompiled from source too in order to be an arm executable, on par with the underlying arm ubuntu image? I find the error where python is not found on the docker images really strange, since that should work out of the box due to docker. Am I missing something obvious here? Again, if anyone has something to propose it would help a lot, Regards.

Hey, I believe I’ve run into this issue before on my Apple machine and have a solution. I am running on the assumption here your Python installation is 3.11. That’s the only reason I can see as to why you’d have this problem (this email may be pretty pointless if this assumption isn’t true). In short: Python 3.11 is incompatible with gem5 in a few ways in gem5 v22.1. Python 3.11 just so happens to be the default installation via Homebrew for the latest Mac hardware. So you should roll-back to an earlier version of Python (Python 3.10 is known to work fine). I’ve also found there’s a small bug if you try to compile anything needing the SPARC ISA on Apple Silicon. If you want to do this, you’ll need to incorporate this patch: https://gem5-review.googlesource.com/c/public/gem5/+/68838. A more in-depth explanation (if anyone is interested): There are three bugs in v22.1 which make it incompatible with Python 3.11. Two are fixed but only exist on the develop branch. The other I’ve yet to find a good solution to. - Bug 1: ‘getargspec’ has been removed in Python 3.11. Fixed with: https://gem5-review.googlesource.com/c/public/gem5/+/68817 - Bug 2: gem5 v22.1 (and earlier) uses a version of Pybind which is incompatible with Python 3.11. This can be fixed by updating “ext/pybind11” to the latest Pybind11 version (Note: the error received with this bug is an unhelpful “Can’t find a Working Python Installation” error, it took me a while to diagnose). The specific patch that fixes this on our develop branch is: https://gem5-review.googlesource.com/c/public/gem5/+/68818 - Bug 3: This bug is explained, in detail, here https://gem5.atlassian.net/browse/GEM5-1321. It relates to an incompatibility with Ply (or, at least, how gem5 uses it) and Python 3.11’s stricter rules on Regex Parsing. At present there is no fix for this bug. Kind regards, Bobby -- Kind regards, Bobby -- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net > On Apr 3, 2023, at 6:36 AM, chcarcher--- via gem5-dev <gem5-dev@gem5.org> wrote: > > Dear all, > > I am trying to build gem5 in an (arm) mac machine and I am encountering some problems. If anyone has managed to do so, I would be grateful if they could share some pointers. > > Here are the various methods that I have tried so far: > > Using the pre-built docker images (gcr.io/gem5-test/ubuntu*) I can successfully build and run gem5, but since the images are build for x86, recompilation takes really long time. > When trying to build natively on apple silicon, (python3 and python development headers/python3-config installed from brew), scons complains about not finding a working python installation. If could someone write where the log files of scons are, I might be able to dig deeper. > > I have tried also re-building the docker images on mac (using the dockerfiles from repo) because this way the underline ubuntu image is an ARM one and emulation of x86 on docker won't be needed. Unfortunately: > On ubuntu 20 with gcc-10 and ubuntu 20 with all dependencies images, scons again reports not finding a working python installation > On ubuntu 22 with all dependencies, looks like there is an incompatibility among protobuf version and the version required by the headers, so compilation throws an error. > On ubuntu 20 with minimum dependencies image, gem builds successfully but throws an error on runtime since it tries to access the (dynamically linked? ) qemu-x86_64 (qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory). Could this be recompiled from source too in order to be an arm executable, on par with the underlying arm ubuntu image? > I find the error where python is not found on the docker images really strange, since that should work out of the box due to docker. Am I missing something obvious here? > > Again, if anyone has something to propose it would help a lot, > Regards. > > > > _______________________________________________ > gem5-dev mailing list -- gem5-dev@gem5.org > To unsubscribe send an email to gem5-dev-leave@gem5.org

Also, I don’t fully understand the docker issues you are facing. Could you send us the actual docker commands you are running so I can try to reproduce the error? The images we provide should work (perhaps there is some incompatibility with running them on Mac, but if this is the case I’ve yet to be be made aware of it). For #5, are you building and running in the same docker container? -- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net > On Apr 3, 2023, at 12:59 PM, Bobby Bruce <bbruce@ucdavis.edu> wrote: > > Hey, > > I believe I’ve run into this issue before on my Apple machine and have a solution. I am running on the assumption here your Python installation is 3.11. That’s the only reason I can see as to why you’d have this problem (this email may be pretty pointless if this assumption isn’t true). In short: Python 3.11 is incompatible with gem5 in a few ways in gem5 v22.1. Python 3.11 just so happens to be the default installation via Homebrew for the latest Mac hardware. So you should roll-back to an earlier version of Python (Python 3.10 is known to work fine). > > I’ve also found there’s a small bug if you try to compile anything needing the SPARC ISA on Apple Silicon. If you want to do this, you’ll need to incorporate this patch: https://gem5-review.googlesource.com/c/public/gem5/+/68838. > > > A more in-depth explanation (if anyone is interested): > > There are three bugs in v22.1 which make it incompatible with Python 3.11. Two are fixed but only exist on the develop branch. The other I’ve yet to find a good solution to. > > - Bug 1: ‘getargspec’ has been removed in Python 3.11. Fixed with: https://gem5-review.googlesource.com/c/public/gem5/+/68817 > - Bug 2: gem5 v22.1 (and earlier) uses a version of Pybind which is incompatible with Python 3.11. This can be fixed by updating “ext/pybind11” to the latest Pybind11 version (Note: the error received with this bug is an unhelpful “Can’t find a Working Python Installation” error, it took me a while to diagnose). The specific patch that fixes this on our develop branch is: https://gem5-review.googlesource.com/c/public/gem5/+/68818 > - Bug 3: This bug is explained, in detail, here https://gem5.atlassian.net/browse/GEM5-1321. It relates to an incompatibility with Ply (or, at least, how gem5 uses it) and Python 3.11’s stricter rules on Regex Parsing. At present there is no fix for this bug. > > Kind regards, > Bobby > > -- > Kind regards, > Bobby > -- > Dr. Bobby R. Bruce > Room 3050, > Kemper Hall, UC Davis > Davis, > CA, 95616 > > web: https://www.bobbybruce.net > >> On Apr 3, 2023, at 6:36 AM, chcarcher--- via gem5-dev <gem5-dev@gem5.org> wrote: >> >> Dear all, >> >> I am trying to build gem5 in an (arm) mac machine and I am encountering some problems. If anyone has managed to do so, I would be grateful if they could share some pointers. >> >> Here are the various methods that I have tried so far: >> >> Using the pre-built docker images (gcr.io/gem5-test/ubuntu*) I can successfully build and run gem5, but since the images are build for x86, recompilation takes really long time. >> When trying to build natively on apple silicon, (python3 and python development headers/python3-config installed from brew), scons complains about not finding a working python installation. If could someone write where the log files of scons are, I might be able to dig deeper. >> >> I have tried also re-building the docker images on mac (using the dockerfiles from repo) because this way the underline ubuntu image is an ARM one and emulation of x86 on docker won't be needed. Unfortunately: >> On ubuntu 20 with gcc-10 and ubuntu 20 with all dependencies images, scons again reports not finding a working python installation >> On ubuntu 22 with all dependencies, looks like there is an incompatibility among protobuf version and the version required by the headers, so compilation throws an error. >> On ubuntu 20 with minimum dependencies image, gem builds successfully but throws an error on runtime since it tries to access the (dynamically linked? ) qemu-x86_64 (qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory). Could this be recompiled from source too in order to be an arm executable, on par with the underlying arm ubuntu image? >> I find the error where python is not found on the docker images really strange, since that should work out of the box due to docker. Am I missing something obvious here? >> >> Again, if anyone has something to propose it would help a lot, >> Regards. >> >> >> >> _______________________________________________ >> gem5-dev mailing list -- gem5-dev@gem5.org >> To unsubscribe send an email to gem5-dev-leave@gem5.org >

Hello and thank you for your reply, For the native compilation I was indeed using python 3.11 and downgrading to 3.9/3.10 enabled me to build/run successfully! For the dockers, "cannot find working installation" error might have been arising from (accidentally) sharing the same volume between multiple builds in different containers (possibly in native build too), since with a fresh pull of gem5 src in a different location I am able to build it in all the images. I didn't check if the error I describe on #5 still remains or if it was also a result of sharing volumes, but if you want to confirm that, let me know and I can see if it is reproducible. Once again, thanks for your help. Regards.