Infinite recursion

[Harmon758]

There is currently an issue with infinite recursion that occurs with the latest version of python-dice (v2.4.0), using Python 3.7 and pyparsing 2.4.0, which does not occur when using pyparsing 2.3.1.

Simply executing dice.roll('6') will immediately cause the program to start using up all available memory, potentially causing the system to freeze.
If a keyboard interrupt is used to stop the execution, the traceback looks like this:

>>> dice.roll('6')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 24, in roll
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 43, in _roll
    ast = parse_expression(string)
  File "C:\Program Files\Python37\lib\site-packages\dice\__init__.py", line 38, in parse_expression
    return dice.grammar.expression.parseString(string, parseAll=True)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 1811, in parseString
    self.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  
  ...
  
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4651, in streamline
    self.expr.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3892, in streamline
    super(MatchFirst, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3655, in streamline
    e.streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3728, in streamline
    super(And, self).streamline()
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3681, in streamline
    self.errmsg = "Expected " + _ustr(self)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  
  ...
    
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in <genexpr>
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in __str__
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3774, in <genexpr>
    self.strRepr = "{" + " ".join(_ustr(e) for e in self.exprs) + "}"
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 4671, in __str__
    retString = _ustr(self.expr)
  File "C:\Program Files\Python37\lib\site-packages\pyparsing.py", line 3931, in __str__
    self.strRepr = "{" + " | ".join(_ustr(e) for e in self.exprs) + "}"
KeyboardInterrupt

The relevant expression can be seen at https://github.com/borntyping/python-dice/blob/v2.4.0/dice/grammar.py.

I believe this is due to the change in ed2f5ec (#71).
Edit: I've now tested this and the issue does not occur when this change is reverted.

cc @calebj

[calebj]

My initial theory is that our custom operatorPrecedence facility creates a cycle in Forward, but I haven't had time to confirm that. We probably need to move to the provided infixNotation eventually.

For now, I've released python-dice 2.4.1 which pins pyparsing to <2.4.0.

[ptmcg]

I downloaded python-dice and looked at the the custom operatorPrecedence code, but nothing jumps out as a left-recursion (which is usually the source of infinite recursion issues in pyparsing). I did a brief experiment replacing the custom operatorPrecedence with pyparsing's infixNotation, and this appeared to work properly, but I did not do a rigorous test.

[ptmcg]

Please check latest update of pyparsing.py to see if this addresses or at least mitigates this issue.

[Harmon758]

I assume by latest update you mean 0ca1706.
I've tested that commit and the issue persists.

[ptmcg]

Thanks - I'll keep looking at this, it's the last major bug I want to fix before pushing out 2.4.1

[ptmcg]

I tested this latest change against my own copy of python-dice and it looks like things are fixed. I also think you can remove your disabling of trim_arity, since the TypeError masking behavior was resolved in 2.1.8 - August, 2016.

[Harmon758]

I can confirm that it seems to be fixed with c5c4ca6.
I'm not a maintainer for python-dice, but I believe you already mentioned that in borntyping/python-dice#23.

[ptmcg]

Fixed in 2.4.1, released today