gem5-dev@gem5.org

The gem5 Developer List

View all threads

[S] Change in gem5/gem5[develop]: arch: set multiline re as default in isa_parser

YW
Yu-hsin Wang (Gerrit)
Thu, Jun 15, 2023 10:04 AM

Yu-hsin Wang has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/71019?usp=email )

Change subject: arch: set multiline re as default in isa_parser
......................................................................

arch: set multiline re as default in isa_parser

In python3.11, it requires the global specifier should be the first
token of regex. However it's not possible when using ply library.
Instead, we set the rules are multiline regex by default and modifies
those single line rules.

Ref: https://github.com/dabeaz/ply/issues/282

Change-Id: I7bdbfeb97a9dd74f45c1890a76f8cc16100e5a42
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71019
Reviewed-by: Richard Cooper richard.cooper@arm.com
Tested-by: kokoro noreply+kokoro@google.com
Maintainer: Jason Lowe-Power power.jg@gmail.com
Reviewed-by: Jason Lowe-Power power.jg@gmail.com

M src/arch/isa_parser/isa_parser.py
1 file changed, 6 insertions(+), 5 deletions(-)

Approvals:
Jason Lowe-Power: Looks good to me, approved; Looks good to me, approved
kokoro: Regressions pass
Richard Cooper: Looks good to me, but someone else must approve

diff --git a/src/arch/isa_parser/isa_parser.py
b/src/arch/isa_parser/isa_parser.py
index 0f29840..5be50a1 100755
--- a/src/arch/isa_parser/isa_parser.py
+++ b/src/arch/isa_parser/isa_parser.py
@@ -514,6 +514,7 @@
class ISAParser(Grammar):
def init(self, output_dir):
super().init()

  •    self.lex_kwargs["reflags"] = int(re.MULTILINE)
        self.output_dir = output_dir
    
        self.filename = None  # for output file watermarking/scaremongering
    

@@ -851,7 +852,7 @@
# String literal.  Note that these use only single quotes, and
# can span multiple lines.
def t_STRLIT(self, t):

  •    r"(?m)'([^'])+'"
    
  •    r"'([^'])+'"
        # strip off quotes
        t.value = t.value[1:-1]
        t.lexer.lineno += t.value.count("\n")
    

@@ -860,19 +861,19 @@
# "Code literal"... like a string literal, but delimiters are
# '{{' and '}}' so they get formatted nicely under emacs c-mode
def t_CODELIT(self, t):

  •    r"(?m)\{\{([^\}]|}(?!\}))+\}\}"
    
  •    r"\{\{([^\}]|}(?!\}))+\}\}"
        # strip off {{ & }}
        t.value = t.value[2:-2]
        t.lexer.lineno += t.value.count("\n")
        return t
    
    def t_CPPDIRECTIVE(self, t):
    
  •    r"^\#[^\#].*\n"
    
  •    r"^\#[^\#][^\n]*\n"
        t.lexer.lineno += t.value.count("\n")
        return t
    
    def t_NEWFILE(self, t):
    
  •    r'^\#\#newfile\s+"[^"]*"\n'
    
  •    r'^\#\#newfile\s+"[^"\n]*"\n'
        self.fileNameStack.push(t.lexer.lineno)
        t.lexer.lineno = LineTracker(t.value[11:-2])
    

@@ -892,7 +893,7 @@

  # Comments
  def t_comment(self, t):
  •    r"//.*"
    
  •    r"//[^\n]*\n"
    
    # Completely ignored characters
    t_ignore = " \t\x0c"
    

--
To view, visit
https://gem5-review.googlesource.com/c/public/gem5/+/71019?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gem5-review.googlesource.com/settings?usp=email

Gerrit-MessageType: merged
Gerrit-Project: public/gem5
Gerrit-Branch: develop
Gerrit-Change-Id: I7bdbfeb97a9dd74f45c1890a76f8cc16100e5a42
Gerrit-Change-Number: 71019
Gerrit-PatchSet: 3
Gerrit-Owner: Yu-hsin Wang yuhsingw@google.com
Gerrit-Reviewer: Bobby Bruce bbruce@ucdavis.edu
Gerrit-Reviewer: Gabe Black gabe.black@gmail.com
Gerrit-Reviewer: Gabe Black gabeblack@google.com
Gerrit-Reviewer: Jason Lowe-Power power.jg@gmail.com
Gerrit-Reviewer: Richard Cooper richard.cooper@arm.com
Gerrit-Reviewer: Yu-hsin Wang yuhsingw@google.com
Gerrit-Reviewer: kokoro noreply+kokoro@google.com
Gerrit-CC: Earl Ou shunhsingou@google.com
Gerrit-CC: kokoro noreply+kokoro@google.com

Yu-hsin Wang has submitted this change. ( https://gem5-review.googlesource.com/c/public/gem5/+/71019?usp=email ) Change subject: arch: set multiline re as default in isa_parser ...................................................................... arch: set multiline re as default in isa_parser In python3.11, it requires the global specifier should be the first token of regex. However it's not possible when using ply library. Instead, we set the rules are multiline regex by default and modifies those single line rules. Ref: https://github.com/dabeaz/ply/issues/282 Change-Id: I7bdbfeb97a9dd74f45c1890a76f8cc16100e5a42 Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/71019 Reviewed-by: Richard Cooper <richard.cooper@arm.com> Tested-by: kokoro <noreply+kokoro@google.com> Maintainer: Jason Lowe-Power <power.jg@gmail.com> Reviewed-by: Jason Lowe-Power <power.jg@gmail.com> --- M src/arch/isa_parser/isa_parser.py 1 file changed, 6 insertions(+), 5 deletions(-) Approvals: Jason Lowe-Power: Looks good to me, approved; Looks good to me, approved kokoro: Regressions pass Richard Cooper: Looks good to me, but someone else must approve diff --git a/src/arch/isa_parser/isa_parser.py b/src/arch/isa_parser/isa_parser.py index 0f29840..5be50a1 100755 --- a/src/arch/isa_parser/isa_parser.py +++ b/src/arch/isa_parser/isa_parser.py @@ -514,6 +514,7 @@ class ISAParser(Grammar): def __init__(self, output_dir): super().__init__() + self.lex_kwargs["reflags"] = int(re.MULTILINE) self.output_dir = output_dir self.filename = None # for output file watermarking/scaremongering @@ -851,7 +852,7 @@ # String literal. Note that these use only single quotes, and # can span multiple lines. def t_STRLIT(self, t): - r"(?m)'([^'])+'" + r"'([^'])+'" # strip off quotes t.value = t.value[1:-1] t.lexer.lineno += t.value.count("\n") @@ -860,19 +861,19 @@ # "Code literal"... like a string literal, but delimiters are # '{{' and '}}' so they get formatted nicely under emacs c-mode def t_CODELIT(self, t): - r"(?m)\{\{([^\}]|}(?!\}))+\}\}" + r"\{\{([^\}]|}(?!\}))+\}\}" # strip off {{ & }} t.value = t.value[2:-2] t.lexer.lineno += t.value.count("\n") return t def t_CPPDIRECTIVE(self, t): - r"^\#[^\#].*\n" + r"^\#[^\#][^\n]*\n" t.lexer.lineno += t.value.count("\n") return t def t_NEWFILE(self, t): - r'^\#\#newfile\s+"[^"]*"\n' + r'^\#\#newfile\s+"[^"\n]*"\n' self.fileNameStack.push(t.lexer.lineno) t.lexer.lineno = LineTracker(t.value[11:-2]) @@ -892,7 +893,7 @@ # Comments def t_comment(self, t): - r"//.*" + r"//[^\n]*\n" # Completely ignored characters t_ignore = " \t\x0c" -- To view, visit https://gem5-review.googlesource.com/c/public/gem5/+/71019?usp=email To unsubscribe, or for help writing mail filters, visit https://gem5-review.googlesource.com/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: public/gem5 Gerrit-Branch: develop Gerrit-Change-Id: I7bdbfeb97a9dd74f45c1890a76f8cc16100e5a42 Gerrit-Change-Number: 71019 Gerrit-PatchSet: 3 Gerrit-Owner: Yu-hsin Wang <yuhsingw@google.com> Gerrit-Reviewer: Bobby Bruce <bbruce@ucdavis.edu> Gerrit-Reviewer: Gabe Black <gabe.black@gmail.com> Gerrit-Reviewer: Gabe Black <gabeblack@google.com> Gerrit-Reviewer: Jason Lowe-Power <power.jg@gmail.com> Gerrit-Reviewer: Richard Cooper <richard.cooper@arm.com> Gerrit-Reviewer: Yu-hsin Wang <yuhsingw@google.com> Gerrit-Reviewer: kokoro <noreply+kokoro@google.com> Gerrit-CC: Earl Ou <shunhsingou@google.com> Gerrit-CC: kokoro <noreply+kokoro@google.com>