gem5-dev@gem5.org

The gem5 Developer List

View all threads

[S] Change in gem5/gem5[develop]: resources: Catch ConnectionResourceError in downloading resources

MJ
Melissa Jost (Gerrit)
Wed, Jun 28, 2023 4:20 PM

Melissa Jost has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/71858?usp=email )

Change subject: resources: Catch ConnectionResourceError in downloading
resources
......................................................................

resources: Catch ConnectionResourceError in downloading resources

This handles an error we see within GitHub Actions that
occassionally occurs when downloading resources.  We retry in the
same way we do when handling HTTPErrors.

Change-Id: I4dce5d607ccc41ad53b51e39082c486e644d815c
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71858
Tested-by: kokoro noreply+kokoro@google.com
Reviewed-by: Jason Lowe-Power power.jg@gmail.com
Maintainer: Jason Lowe-Power power.jg@gmail.com

M src/python/gem5/resources/downloader.py
1 file changed, 16 insertions(+), 0 deletions(-)

Approvals:
kokoro: Regressions pass
Jason Lowe-Power: Looks good to me, approved; Looks good to me, approved

diff --git a/src/python/gem5/resources/downloader.py
b/src/python/gem5/resources/downloader.py
index bb5ca84..d606f34 100644
--- a/src/python/gem5/resources/downloader.py
+++ b/src/python/gem5/resources/downloader.py
@@ -139,6 +139,22 @@
time.sleep((2**attempt) + random.uniform(0, 1))
else:
raise e

  •    except ConnectionResetError as e:
    
  •        # This catches the ConnectionResetError we see occassionally  
    

see

  •        # when accessing resources on GitHub Actions.  It retries  
    

using a

  •        # Truncated Exponential backoff algorithm, truncating after
    
  •        # "max_attempts". If any other is retrieved we raise the error.
    
  •        if e.errno == 104:
    
  •            attempt += 1
    
  •            if attempt >= max_attempts:
    
  •                raise Exception(
    
  •                    f"After {attempt} attempts, the resource json  
    

could "

  •                    "not be retrieved. OS Error Code retrieved: "
    
  •                    f"{e.errno}"
    
  •                )
    
  •            time.sleep((2**attempt) + random.uniform(0, 1))
    
  •        else:
    
  •            raise e
        except ValueError as e:
            raise Exception(
                "Environment variable GEM5_USE_PROXY is set to "
    

--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/71858?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings?usp=email

Gerrit-MessageType: merged
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I4dce5d607ccc41ad53b51e39082c486e644d815c
Gerrit-Change-Number: 71858
Gerrit-PatchSet: 2
Gerrit-Owner: Melissa Jost melissakjost@gmail.com
Gerrit-Reviewer: Bobby Bruce bbruce@ucdavis.edu
Gerrit-Reviewer: Jason Lowe-Power jason@lowepower.com
Gerrit-Reviewer: Jason Lowe-Power power.jg@gmail.com
Gerrit-Reviewer: Melissa Jost melissakjost@gmail.com
Gerrit-Reviewer: kokoro noreply+kokoro@google.com
Gerrit-CC: kokoro noreply+kokoro@google.com

Melissa Jost has submitted this change. ( https://gem5-review.googlesource.com/c/public/gem5/+/71858?usp=email ) Change subject: resources: Catch ConnectionResourceError in downloading resources ...................................................................... resources: Catch ConnectionResourceError in downloading resources This handles an error we see within GitHub Actions that occassionally occurs when downloading resources. We retry in the same way we do when handling HTTPErrors. Change-Id: I4dce5d607ccc41ad53b51e39082c486e644d815c Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71858 Tested-by: kokoro <noreply+kokoro@google.com> Reviewed-by: Jason Lowe-Power <power.jg@gmail.com> Maintainer: Jason Lowe-Power <power.jg@gmail.com> --- M src/python/gem5/resources/downloader.py 1 file changed, 16 insertions(+), 0 deletions(-) Approvals: kokoro: Regressions pass Jason Lowe-Power: Looks good to me, approved; Looks good to me, approved diff --git a/src/python/gem5/resources/downloader.py b/src/python/gem5/resources/downloader.py index bb5ca84..d606f34 100644 --- a/src/python/gem5/resources/downloader.py +++ b/src/python/gem5/resources/downloader.py @@ -139,6 +139,22 @@ time.sleep((2**attempt) + random.uniform(0, 1)) else: raise e + except ConnectionResetError as e: + # This catches the ConnectionResetError we see occassionally see + # when accessing resources on GitHub Actions. It retries using a + # Truncated Exponential backoff algorithm, truncating after + # "max_attempts". If any other is retrieved we raise the error. + if e.errno == 104: + attempt += 1 + if attempt >= max_attempts: + raise Exception( + f"After {attempt} attempts, the resource json could " + "not be retrieved. OS Error Code retrieved: " + f"{e.errno}" + ) + time.sleep((2**attempt) + random.uniform(0, 1)) + else: + raise e except ValueError as e: raise Exception( "Environment variable GEM5_USE_PROXY is set to " -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/71858?usp=email To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: public/gem5 Gerrit-Branch: develop Gerrit-Change-Id: I4dce5d607ccc41ad53b51e39082c486e644d815c Gerrit-Change-Number: 71858 Gerrit-PatchSet: 2 Gerrit-Owner: Melissa Jost <melissakjost@gmail.com> Gerrit-Reviewer: Bobby Bruce <bbruce@ucdavis.edu> Gerrit-Reviewer: Jason Lowe-Power <jason@lowepower.com> Gerrit-Reviewer: Jason Lowe-Power <power.jg@gmail.com> Gerrit-Reviewer: Melissa Jost <melissakjost@gmail.com> Gerrit-Reviewer: kokoro <noreply+kokoro@google.com> Gerrit-CC: kokoro <noreply+kokoro@google.com>