gem5-dev@gem5.org

The gem5 Developer List

View all threads

[M] Change in gem5/gem5[stable]: misc: Add weekly tests to Github workflows

MJ
Melissa Jost (Gerrit)
Fri, Jun 9, 2023 2:18 AM

Melissa Jost has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/71401?usp=email )

Change subject: misc: Add weekly tests to Github workflows
......................................................................

misc: Add weekly tests to Github workflows

This introduces the very-long tests from our weekly tests to be
able to run within GitHub.

Change-Id: I5f5b4c521b47272cb27640c572a869c26c04b95a
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71401
Reviewed-by: Bobby Bruce bbruce@ucdavis.edu
Maintainer: Bobby Bruce bbruce@ucdavis.edu
Tested-by: kokoro noreply+kokoro@google.com

A .github/workflows/weekly-tests.yaml
1 file changed, 178 insertions(+), 0 deletions(-)

Approvals:
Bobby Bruce: Looks good to me, approved; Looks good to me, approved
kokoro: Regressions pass

diff --git a/.github/workflows/weekly-tests.yaml
b/.github/workflows/weekly-tests.yaml
new file mode 100644
index 0000000..48fb79f
--- /dev/null
+++ b/.github/workflows/weekly-tests.yaml
@@ -0,0 +1,178 @@
+# This workflow runs all of the very-long tests within main.py
+
+name: Running weekly main.py
+run-name: ${{ github.actor }} is running the weekly tests
+
+
+on:

  • Runs every Sunday from 7AM UTC

  • schedule:
    • cron:  '00 7 * * 6'
  • Allows us to manually start workflow for testing

  • workflow_dispatch:

+jobs:

  • build-gem5:
  • runs-on: [self-hosted, linux, x64, build]
  • container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest
  • outputs:
  •  build-name: ${{ steps.artifact-name.outputs.name }}
    
  • steps:
  •  - uses: actions/checkout@v3
    
  •    with:
    
  •      # Scheduled workflows run on the default branch by default. We
    
  •      # therefore need to explicitly checkout the develop branch.
    
  •      ref: develop
    
  •  - id: artifact-name
    
  •    run: echo "name=$(date +"%Y-%m-%d_%H.%M.%S")-ALL" >> $GITHUB_OUTPUT
    
  •  - name: Build gem5
    
  •    run: |
    
  •      scons build/ALL/gem5.opt -j $(nproc)
    
  •  - uses: actions/upload-artifact@v3
    
  •    with:
    
  •      name: ${{ steps.artifact-name.outputs.name }}
    
  •      path: build/ALL/gem5.opt
    
  •      retention-days: 5
    
  •  - run: echo "This job's status is ${{ job.status }}."
    

+# start running the very-long tests

  • testlib-very-long-gem5_library_example_tests:
  • runs-on: [self-hosted, linux, x64, run]
  • container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest
  • needs: [build-gem5]
  • timeout-minutes: 1440 # 24 hours
  • steps:
    • uses: actions/checkout@v3
  •  with:
    
  •    # Scheduled workflows run on the default branch by default. We
    
  •    # therefore need to explicitly checkout the develop branch.
    
  •    ref: develop
    
    • uses: actions/download-artifact@v3
  •  with:
    
  •    name: ${{needs.build-gem5.outputs.build-name}}
    
  •    path: build/ALL
    
    • run: chmod u+x build/ALL/gem5.opt
    • name: very-long gem5_library_example_tests
  •  working-directory: ${{ github.workspace }}/tests
    
  •  run: ./main.py run gem5/gem5_library_example_tests --length  
    

very-long --skip-build -vv -t $(nproc)

    • name: create zip of results
  •  if: success() || failure()
    
  •  run: |
    
  •    apt-get -y install zip
    
  •    zip -r output.zip tests/testing-results
    
    • name: upload zip
  •  if: success() || failure()
    
  •  uses: actions/upload-artifact@v3
    
  •  env:
    
  •    MY_STEP_VAR:  
    

${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}}

  •  with:
    
  •    name: ${{ env.MY_STEP_VAR }}
    
  •    path: output.zip
    
  •    retention-days: 7
    
    • run: echo "This job's status is ${{ job.status }}."
  • testlib-long-gem5-resources:
  • runs-on: [self-hosted, linux, x64, run]
  • container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest
  • needs: [build-gem5]
  • timeout-minutes: 1440 # 24 hours
  • steps:
    • uses: actions/checkout@v3
  •  with:
    
  •    # Scheduled workflows run on the default branch by default. We
    
  •    # therefore need to explicitly checkout the develop branch.
    
  •    ref: develop
    
    • uses: actions/download-artifact@v3
  •  with:
    
  •    name: ${{needs.build-gem5.outputs.build-name}}
    
  •    path: build/ALL
    
    • run: chmod u+x build/ALL/gem5.opt
    • name: very-long gem5-resources tests
  •  working-directory: ${{ github.workspace }}/tests
    
  •  run: ./main.py run gem5/gem5-resources --length very-long  
    

--skip-build -vv -t $(nproc)

    • name: create zip of results
  •  if: success() || failure()
    
  •  run: |
    
  •    apt-get -y install zip
    
  •    zip -r output.zip tests/testing-results
    
    • name: upload zip
  •  if: success() || failure()
    
  •  uses: actions/upload-artifact@v3
    
  •  env:
    
  •    MY_STEP_VAR:  
    

${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}}

  •  with:
    
  •    name: ${{ env.MY_STEP_VAR }}
    
  •    path: output.zip
    
  •    retention-days: 7
    
    • run: echo "This job's status is ${{ job.status }}."
  • testlib-very-long-parsec-benchmarks:
  • runs-on: [self-hosted, linux, x64, run]
  • container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest
  • needs: build-gem5
  • timeout-minutes: 1440 # 24 hours
  • steps:
    • uses: actions/checkout@v3
  •  with:
    
  •    # Scheduled workflows run on the default branch by default. We
    
  •    # therefore need to explicitly checkout the develop branch.
    
  •    ref: develop
    
    • uses: actions/download-artifact@v3
  •  with:
    
  •    name: ${{needs.build-gem5.outputs.build-name}}
    
  •    path: build/ALL
    
    • run: chmod u+x build/ALL/gem5.opt
    • name: very-long x86-boot-tests
  •  working-directory: ${{ github.workspace }}/tests
    
  •  run: ./main.py run gem5/x86-boot-tests --length very-long  
    

--skip-build -vv -t $(nproc)

    • name: create zip of results
  •  if: success() || failure()
    
  •  run: |
    
  •    apt-get -y install zip
    
  •    zip -r output.zip tests/testing-results
    
    • name: upload zip
  •  if: success() || failure()
    
  •  uses: actions/upload-artifact@v3
    
  •  env:
    
  •    MY_STEP_VAR:  
    

${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}}

  •  with:
    
  •    name: ${{ env.MY_STEP_VAR }}
    
  •    path: output.zip
    
  •    retention-days: 7
    
    • run: echo "This job's status is ${{ job.status }}."
  • testlib-very-long-x86-boot-tests:
  • runs-on: [self-hosted, linux, x64, run]
  • container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest
  • needs: build-gem5
  • timeout-minutes: 1440 # 24 hours
  • steps:
    • uses: actions/checkout@v3
  •  with:
    
  •    # Scheduled workflows run on the default branch by default. We
    
  •    # therefore need to explicitly checkout the develop branch.
    
  •    ref: develop
    
    • uses: actions/download-artifact@v3
  •  with:
    
  •    name: ${{needs.build-gem5.outputs.build-name}}
    
  •    path: build/ALL
    
    • run: chmod u+x build/ALL/gem5.opt
    • name: very-long x86-boot-tests
  •  working-directory: ${{ github.workspace }}/tests
    
  •  run: ./main.py run gem5/x86-boot-tests --length very-long  
    

--skip-build -vv -t $(nproc)

    • name: create zip of results
  •  if: success() || failure()
    
  •  run: |
    
  •    apt-get -y install zip
    
  •    zip -r output.zip tests/testing-results
    
    • name: upload zip
  •  if: success() || failure()
    
  •  uses: actions/upload-artifact@v3
    
  •  env:
    
  •    MY_STEP_VAR:  
    

${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}}

  •  with:
    
  •    name: ${{ env.MY_STEP_VAR }}
    
  •    path: output.zip
    
  •    retention-days: 7
    
    • run: echo "This job's status is ${{ job.status }}."

--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/71401?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings?usp=email

Gerrit-MessageType: merged
Gerrit-Project: public/gem5
Gerrit-Branch: stable
Gerrit-Change-Id: I5f5b4c521b47272cb27640c572a869c26c04b95a
Gerrit-Change-Number: 71401
Gerrit-PatchSet: 4
Gerrit-Owner: Melissa Jost melissakjost@gmail.com
Gerrit-Reviewer: Bobby Bruce bbruce@ucdavis.edu
Gerrit-Reviewer: Jason Lowe-Power jason@lowepower.com
Gerrit-Reviewer: Melissa Jost mkjost@ucdavis.edu
Gerrit-Reviewer: kokoro noreply+kokoro@google.com
Gerrit-CC: Matt Sinclair mattdsinclair@gmail.com
Gerrit-CC: kokoro noreply+kokoro@google.com

Melissa Jost has submitted this change. ( https://gem5-review.googlesource.com/c/public/gem5/+/71401?usp=email ) Change subject: misc: Add weekly tests to Github workflows ...................................................................... misc: Add weekly tests to Github workflows This introduces the very-long tests from our weekly tests to be able to run within GitHub. Change-Id: I5f5b4c521b47272cb27640c572a869c26c04b95a Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71401 Reviewed-by: Bobby Bruce <bbruce@ucdavis.edu> Maintainer: Bobby Bruce <bbruce@ucdavis.edu> Tested-by: kokoro <noreply+kokoro@google.com> --- A .github/workflows/weekly-tests.yaml 1 file changed, 178 insertions(+), 0 deletions(-) Approvals: Bobby Bruce: Looks good to me, approved; Looks good to me, approved kokoro: Regressions pass diff --git a/.github/workflows/weekly-tests.yaml b/.github/workflows/weekly-tests.yaml new file mode 100644 index 0000000..48fb79f --- /dev/null +++ b/.github/workflows/weekly-tests.yaml @@ -0,0 +1,178 @@ +# This workflow runs all of the very-long tests within main.py + +name: Running weekly main.py +run-name: ${{ github.actor }} is running the weekly tests + + +on: + # Runs every Sunday from 7AM UTC + schedule: + - cron: '00 7 * * 6' + # Allows us to manually start workflow for testing + workflow_dispatch: + +jobs: + build-gem5: + runs-on: [self-hosted, linux, x64, build] + container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest + outputs: + build-name: ${{ steps.artifact-name.outputs.name }} + steps: + - uses: actions/checkout@v3 + with: + # Scheduled workflows run on the default branch by default. We + # therefore need to explicitly checkout the develop branch. + ref: develop + - id: artifact-name + run: echo "name=$(date +"%Y-%m-%d_%H.%M.%S")-ALL" >> $GITHUB_OUTPUT + - name: Build gem5 + run: | + scons build/ALL/gem5.opt -j $(nproc) + - uses: actions/upload-artifact@v3 + with: + name: ${{ steps.artifact-name.outputs.name }} + path: build/ALL/gem5.opt + retention-days: 5 + - run: echo "This job's status is ${{ job.status }}." + +# start running the very-long tests + testlib-very-long-gem5_library_example_tests: + runs-on: [self-hosted, linux, x64, run] + container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest + needs: [build-gem5] + timeout-minutes: 1440 # 24 hours + steps: + - uses: actions/checkout@v3 + with: + # Scheduled workflows run on the default branch by default. We + # therefore need to explicitly checkout the develop branch. + ref: develop + - uses: actions/download-artifact@v3 + with: + name: ${{needs.build-gem5.outputs.build-name}} + path: build/ALL + - run: chmod u+x build/ALL/gem5.opt + - name: very-long gem5_library_example_tests + working-directory: ${{ github.workspace }}/tests + run: ./main.py run gem5/gem5_library_example_tests --length very-long --skip-build -vv -t $(nproc) + - name: create zip of results + if: success() || failure() + run: | + apt-get -y install zip + zip -r output.zip tests/testing-results + - name: upload zip + if: success() || failure() + uses: actions/upload-artifact@v3 + env: + MY_STEP_VAR: ${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}} + with: + name: ${{ env.MY_STEP_VAR }} + path: output.zip + retention-days: 7 + - run: echo "This job's status is ${{ job.status }}." + + + testlib-long-gem5-resources: + runs-on: [self-hosted, linux, x64, run] + container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest + needs: [build-gem5] + timeout-minutes: 1440 # 24 hours + steps: + - uses: actions/checkout@v3 + with: + # Scheduled workflows run on the default branch by default. We + # therefore need to explicitly checkout the develop branch. + ref: develop + - uses: actions/download-artifact@v3 + with: + name: ${{needs.build-gem5.outputs.build-name}} + path: build/ALL + - run: chmod u+x build/ALL/gem5.opt + - name: very-long gem5-resources tests + working-directory: ${{ github.workspace }}/tests + run: ./main.py run gem5/gem5-resources --length very-long --skip-build -vv -t $(nproc) + - name: create zip of results + if: success() || failure() + run: | + apt-get -y install zip + zip -r output.zip tests/testing-results + - name: upload zip + if: success() || failure() + uses: actions/upload-artifact@v3 + env: + MY_STEP_VAR: ${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}} + with: + name: ${{ env.MY_STEP_VAR }} + path: output.zip + retention-days: 7 + - run: echo "This job's status is ${{ job.status }}." + + testlib-very-long-parsec-benchmarks: + runs-on: [self-hosted, linux, x64, run] + container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest + needs: build-gem5 + timeout-minutes: 1440 # 24 hours + steps: + - uses: actions/checkout@v3 + with: + # Scheduled workflows run on the default branch by default. We + # therefore need to explicitly checkout the develop branch. + ref: develop + - uses: actions/download-artifact@v3 + with: + name: ${{needs.build-gem5.outputs.build-name}} + path: build/ALL + - run: chmod u+x build/ALL/gem5.opt + - name: very-long x86-boot-tests + working-directory: ${{ github.workspace }}/tests + run: ./main.py run gem5/x86-boot-tests --length very-long --skip-build -vv -t $(nproc) + - name: create zip of results + if: success() || failure() + run: | + apt-get -y install zip + zip -r output.zip tests/testing-results + - name: upload zip + if: success() || failure() + uses: actions/upload-artifact@v3 + env: + MY_STEP_VAR: ${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}} + with: + name: ${{ env.MY_STEP_VAR }} + path: output.zip + retention-days: 7 + - run: echo "This job's status is ${{ job.status }}." + + testlib-very-long-x86-boot-tests: + runs-on: [self-hosted, linux, x64, run] + container: gcr.io/gem5-test/ubuntu-22.04_all-dependencies:latest + needs: build-gem5 + timeout-minutes: 1440 # 24 hours + steps: + - uses: actions/checkout@v3 + with: + # Scheduled workflows run on the default branch by default. We + # therefore need to explicitly checkout the develop branch. + ref: develop + - uses: actions/download-artifact@v3 + with: + name: ${{needs.build-gem5.outputs.build-name}} + path: build/ALL + - run: chmod u+x build/ALL/gem5.opt + - name: very-long x86-boot-tests + working-directory: ${{ github.workspace }}/tests + run: ./main.py run gem5/x86-boot-tests --length very-long --skip-build -vv -t $(nproc) + - name: create zip of results + if: success() || failure() + run: | + apt-get -y install zip + zip -r output.zip tests/testing-results + - name: upload zip + if: success() || failure() + uses: actions/upload-artifact@v3 + env: + MY_STEP_VAR: ${{github.job}}_COMMIT.${{github.sha}}_RUN.${{github.run_id}}_ATTEMPT.${{github.run_attempt}} + with: + name: ${{ env.MY_STEP_VAR }} + path: output.zip + retention-days: 7 + - run: echo "This job's status is ${{ job.status }}." -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/71401?usp=email To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: public/gem5 Gerrit-Branch: stable Gerrit-Change-Id: I5f5b4c521b47272cb27640c572a869c26c04b95a Gerrit-Change-Number: 71401 Gerrit-PatchSet: 4 Gerrit-Owner: Melissa Jost <melissakjost@gmail.com> Gerrit-Reviewer: Bobby Bruce <bbruce@ucdavis.edu> Gerrit-Reviewer: Jason Lowe-Power <jason@lowepower.com> Gerrit-Reviewer: Melissa Jost <mkjost@ucdavis.edu> Gerrit-Reviewer: kokoro <noreply+kokoro@google.com> Gerrit-CC: Matt Sinclair <mattdsinclair@gmail.com> Gerrit-CC: kokoro <noreply+kokoro@google.com>