pythonparser is not handling print >> filelike correctly

[alanjds]

google#297 opened by @alanjds on 24 Apr 2017

The new AST parser acts strange on print on some case (not all):

# tst.py
import StringIO
sio = StringIO.StringIO()

print 'foo'
print >> sio, 'foo'
print >>sio, 'foo'
print >> sio, 'foo',
print >>sio, 'foo',
print
print >> sio
print >>sio

CPython reference:

$ cat tst.py | python -
foo

$

Before change to pythonparser

$ git checkout c7f97e72452ff91b906c9edabdec0024e9078470
$ cat tst.py | make run
foo
foo
foo
foo foo

$

After moving pythonparser into grumpy

$ git checkout 0def2c25b977bbc200f5797f67ec8ca328893735
$ cat tst.py | make run
<unknown>:10:13-11:1: fatal: unexpected newline: expected !=, %, &, (, *, **, +, -, ., /, //, <, <<, <=, <>, ==, >, >=, >>, @, [, ^, and, if, in, is, not, or or |
print >> sio
            ^
Traceback (most recent call last):
  File "/Users/alanjds/src/git/grumpy/build/bin/grumpc", line 118, in <module>
    sys.exit(main(parser.parse_args()))
  File "/Users/alanjds/src/git/grumpy/build/bin/grumpc", line 53, in main
    mod = pythonparser.parse(py_contents)
  File "/Users/alanjds/src/git/grumpy/build/lib/python2.7/site-packages/grumpy/pythonparser/__init__.py", line 59, in parse
    mode, flags, version, engine)
  File "/Users/alanjds/src/git/grumpy/build/lib/python2.7/site-packages/grumpy/pythonparser/__init__.py", line 29, in parse_buffer
    return parser.file_input(), lexer.comments
  File "/Users/alanjds/src/git/grumpy/build/lib/python2.7/site-packages/grumpy/pythonparser/parser.py", line 89, in outer_rule
    result = inner_rule(parser)
  File "/Users/alanjds/src/git/grumpy/build/lib/python2.7/site-packages/grumpy/pythonparser/parser.py", line 151, in rule
    parser.diagnostic_engine.process(error)
  File "/Users/alanjds/src/git/grumpy/build/lib/python2.7/site-packages/grumpy/pythonparser/diagnostic.py", line 165, in process
    raise Error(diagnostic)
grumpy.pythonparser.diagnostic.Error: <unknown>:10:13-11:1: fatal: unexpected newline: expected !=, %, &, (, *, **, +, -, ., /, //, <, <<, <=, <>, ==, >, >=, >>, @, [, ^, and, if, in, is, not, or or |
print >> sio
            ^
make: *** [run] Error 1

[alanjds]

Comment by trotterdylan
Tuesday Apr 25, 2017 at 03:56 GMT

---

Ah, bummer. Thanks for the report. Would you mind filing this up stream in pythonparser? It should be pretty straightforward to fix.

[alanjds]

Comment by alanjds
Tuesday Apr 25, 2017 at 14:42 GMT

---

No problem. Will post there: http://github.com/m-labs/pythonparser

Nudge me if is the wrong place.