gem5-dev@gem5.org

The gem5 Developer List

View all threads

[M] Change in gem5/gem5[develop]: fastmodel: Check early for license server issue

NB
Nicolas Boichat (Gerrit)
Tue, Mar 7, 2023 11:21 PM

Nicolas Boichat has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/68177?usp=email )

Change subject: fastmodel: Check early for license server issue
......................................................................

fastmodel: Check early for license server issue

We have a setup that requires manual startup of an ssh proxy to
access license server, and without that, gem5 takes about a minute
until the license checkout times out (until then, it's unclear
why nothing is happening).

We asked ARM for a way to decrease timeouts, but that doesn't
seem to be easy to do.

Change-Id: I37b84fd52cb7fb221a9e48dcb52a33a11f4d1580
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/68177
Reviewed-by: Gabe Black gabe.black@gmail.com
Maintainer: Gabe Black gabe.black@gmail.com
Tested-by: kokoro noreply+kokoro@google.com

M src/arch/arm/fastmodel/arm_fast_model.py
1 file changed, 65 insertions(+), 5 deletions(-)

Approvals:
kokoro: Regressions pass
Gabe Black: Looks good to me, approved; Looks good to me, approved

diff --git a/src/arch/arm/fastmodel/arm_fast_model.py
b/src/arch/arm/fastmodel/arm_fast_model.py
index d2d911f..81b2cfe 100644
--- a/src/arch/arm/fastmodel/arm_fast_model.py
+++ b/src/arch/arm/fastmodel/arm_fast_model.py
@@ -23,21 +23,75 @@

(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

+import logging
import os
+import socket

from m5.defines import buildEnv
import _m5.arm_fast_model

+ARM_LICENSE_ENV = "ARMLMD_LICENSE_FILE"
+LM_LICENSE_ENV = "LM_LICENSE_FILE"
+

def set_armlmd_license_file(force=False):
"""Set the ARMLMD_LICENSE_FILE environment variable. If "force" is
False, then it will only be set if it wasn't already set in the
environment. The value it's set to is the one gem5 was built with.
"""

  • key = "ARMLMD_LICENSE_FILE"
  • license_file = buildEnv[key]
  • if force or key not in os.environ:
  •    os.environ[key] = license_file
    
  • license_file = buildEnv[ARM_LICENSE_ENV]
  • if force or ARM_LICENSE_ENV not in os.environ:
  •    os.environ[ARM_LICENSE_ENV] = license_file
    

+def check_armlmd_license(timeout):

  • """Check if any of the provided license server can be reached, or

  • if a license file is provided. This allows to fail early and fast,

  • as fastmodel code makes multiple lengthy attempts to connect to

  • license server. "timeout" is in seconds.

  • """

  • servers = os.environ[ARM_LICENSE_ENV].split(":")

  • extras = list()

  • Add LM_LICENSE_ENV to the list, if set.

  • if LM_LICENSE_ENV in os.environ and os.environ[LM_LICENSE_ENV]:

  •    extras += os.environ[LM_LICENSE_ENV].split(":")
    
  • Fastmodel appears to always add this file.

  • extras.append("/opt/arm/licenses/license.dat")

  • for extra in extras:

  •    if extra not in servers:
    
  •        servers.append(extra)
    
  • for server in servers:

  •    if os.path.exists(server):
    
  •        logging.debug("License file %s exists." % server)
    
  •        break
    
  •    tuple = server.split("@")
    
  •    if len(tuple) != 2:
    
  •        # Probably not a server, and we know the file doesn't exist.
    
  •        logging.debug('License file "%s" does not exist.' % server)
    
  •        continue
    
  •    try:
    
  •        # Try to connect to license server. This doesn't attempt to
    
  •        # communicate with it, just checking reachability.
    
  •        s = socket.create_connection(
    
  •            (tuple[1], int(tuple[0])), timeout=timeout
    
  •        )
    
  •        s.close()
    
  •        logging.debug("License server %s is reachable." % server)
    
  •        break
    
  •    except Exception as e:
    
  •        logging.debug(
    
  •            "Cannot connect to license server %s (%s: %s)."
    
  •            % (server, type(e).__name__, e)
    
  •        )
    
  • else:

  •    raise ConnectionError(
    
  •        "Cannot connect to any of the license servers (%s)."
    
  •        % ", ".join(servers)
    
  •    )
    

    These methods wrap much of the SystemC Export API described in section

@@ -142,9 +196,15 @@

This should be called once per simulation

def setup_simulation(

  • sim_name, min_sync_latency=100.0 / 100000000, exit_on_dmi_warning=False
  • sim_name,
  • min_sync_latency=100.0 / 100000000,
  • exit_on_dmi_warning=False,
  • license_precheck=False,
  • license_precheck_timeout=1,
    ):
    set_armlmd_license_file()
  • if license_precheck:
  •    check_armlmd_license(license_precheck_timeout)
    scx_initialize(sim_name)
    scx_set_min_sync_latency(min_sync_latency)
    if exit_on_dmi_warning:
    

--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/68177?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings

Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I37b84fd52cb7fb221a9e48dcb52a33a11f4d1580
Gerrit-Change-Number: 68177
Gerrit-PatchSet: 9
Gerrit-Owner: Nicolas Boichat drinkcat@google.com
Gerrit-Reviewer: Chia-You Chen hortune@google.com
Gerrit-Reviewer: Gabe Black gabe.black@gmail.com
Gerrit-Reviewer: Nicolas Boichat drinkcat@google.com
Gerrit-Reviewer: Yu-hsin Wang yuhsingw@google.com
Gerrit-Reviewer: kokoro noreply+kokoro@google.com
Gerrit-CC: Gabe Black gabeblack@google.com
Gerrit-MessageType: merged

Nicolas Boichat has submitted this change. ( https://gem5-review.googlesource.com/c/public/gem5/+/68177?usp=email ) Change subject: fastmodel: Check early for license server issue ...................................................................... fastmodel: Check early for license server issue We have a setup that requires manual startup of an ssh proxy to access license server, and without that, gem5 takes about a minute until the license checkout times out (until then, it's unclear why nothing is happening). We asked ARM for a way to decrease timeouts, but that doesn't seem to be easy to do. Change-Id: I37b84fd52cb7fb221a9e48dcb52a33a11f4d1580 Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/68177 Reviewed-by: Gabe Black <gabe.black@gmail.com> Maintainer: Gabe Black <gabe.black@gmail.com> Tested-by: kokoro <noreply+kokoro@google.com> --- M src/arch/arm/fastmodel/arm_fast_model.py 1 file changed, 65 insertions(+), 5 deletions(-) Approvals: kokoro: Regressions pass Gabe Black: Looks good to me, approved; Looks good to me, approved diff --git a/src/arch/arm/fastmodel/arm_fast_model.py b/src/arch/arm/fastmodel/arm_fast_model.py index d2d911f..81b2cfe 100644 --- a/src/arch/arm/fastmodel/arm_fast_model.py +++ b/src/arch/arm/fastmodel/arm_fast_model.py @@ -23,21 +23,75 @@ # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +import logging import os +import socket from m5.defines import buildEnv import _m5.arm_fast_model +ARM_LICENSE_ENV = "ARMLMD_LICENSE_FILE" +LM_LICENSE_ENV = "LM_LICENSE_FILE" + def set_armlmd_license_file(force=False): """Set the ARMLMD_LICENSE_FILE environment variable. If "force" is False, then it will only be set if it wasn't already set in the environment. The value it's set to is the one gem5 was built with. """ - key = "ARMLMD_LICENSE_FILE" - license_file = buildEnv[key] - if force or key not in os.environ: - os.environ[key] = license_file + license_file = buildEnv[ARM_LICENSE_ENV] + if force or ARM_LICENSE_ENV not in os.environ: + os.environ[ARM_LICENSE_ENV] = license_file + + +def check_armlmd_license(timeout): + """Check if any of the provided license server can be reached, or + if a license file is provided. This allows to fail early and fast, + as fastmodel code makes multiple lengthy attempts to connect to + license server. "timeout" is in seconds. + """ + servers = os.environ[ARM_LICENSE_ENV].split(":") + + extras = list() + # Add LM_LICENSE_ENV to the list, if set. + if LM_LICENSE_ENV in os.environ and os.environ[LM_LICENSE_ENV]: + extras += os.environ[LM_LICENSE_ENV].split(":") + # Fastmodel appears to always add this file. + extras.append("/opt/arm/licenses/license.dat") + for extra in extras: + if extra not in servers: + servers.append(extra) + + for server in servers: + if os.path.exists(server): + logging.debug("License file %s exists." % server) + break + + tuple = server.split("@") + if len(tuple) != 2: + # Probably not a server, and we know the file doesn't exist. + logging.debug('License file "%s" does not exist.' % server) + continue + + try: + # Try to connect to license server. This doesn't attempt to + # communicate with it, just checking reachability. + s = socket.create_connection( + (tuple[1], int(tuple[0])), timeout=timeout + ) + s.close() + logging.debug("License server %s is reachable." % server) + break + except Exception as e: + logging.debug( + "Cannot connect to license server %s (%s: %s)." + % (server, type(e).__name__, e) + ) + else: + raise ConnectionError( + "Cannot connect to any of the license servers (%s)." + % ", ".join(servers) + ) # These methods wrap much of the SystemC Export API described in section @@ -142,9 +196,15 @@ # This should be called once per simulation def setup_simulation( - sim_name, min_sync_latency=100.0 / 100000000, exit_on_dmi_warning=False + sim_name, + min_sync_latency=100.0 / 100000000, + exit_on_dmi_warning=False, + license_precheck=False, + license_precheck_timeout=1, ): set_armlmd_license_file() + if license_precheck: + check_armlmd_license(license_precheck_timeout) scx_initialize(sim_name) scx_set_min_sync_latency(min_sync_latency) if exit_on_dmi_warning: -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/68177?usp=email To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings Gerrit-Project: public/gem5 Gerrit-Branch: develop Gerrit-Change-Id: I37b84fd52cb7fb221a9e48dcb52a33a11f4d1580 Gerrit-Change-Number: 68177 Gerrit-PatchSet: 9 Gerrit-Owner: Nicolas Boichat <drinkcat@google.com> Gerrit-Reviewer: Chia-You Chen <hortune@google.com> Gerrit-Reviewer: Gabe Black <gabe.black@gmail.com> Gerrit-Reviewer: Nicolas Boichat <drinkcat@google.com> Gerrit-Reviewer: Yu-hsin Wang <yuhsingw@google.com> Gerrit-Reviewer: kokoro <noreply+kokoro@google.com> Gerrit-CC: Gabe Black <gabeblack@google.com> Gerrit-MessageType: merged