`ast.parse` raises `SystemError: <class 'UnicodeDecodeError'> returned a result with an error set`

[gabe-sherman]

Bug report

Bug description:

The below code triggers the following exception message: SystemError: <class 'UnicodeDecodeError'> returned a result with an error set. On later Python versions(>3.9), this shows up as just a UnicodeDecodeError, so it seems like the issue does not occur in later versions. I tried checking to see if this was known to exist in version 3.9, but didn't see anything, so I am reporting it here. Note that the type_comments argument must be set to True in order for this to occur.

Reproducer

import ast 
d = b'def hello(*onRlionRRasR,#type:RR,rRBRliRRf, o:\n f, o:\n ionRleloionRRasR,#type:RR,rRBRl\x8fRRf, o:\n cRRRlction= la:::::\x1c:\xad\xa2"\n'
parseval1 = ast.parse(d, type_comments = True)

Trace

Traceback (most recent call last):
  File "rep.py", line 5, in <module>
    ast.parse(d, type_comments = True)
  File "/usr/lib/python3.9/ast.py", line 50, in parse
    return compile(source, filename, mode, flags,
SystemError: <class 'UnicodeDecodeError'> returned a result with an error set

CPython versions tested on:

3.9

Operating systems tested on:

Linux

[ronaldoussoren]

3.9 only receives security fixes at this time and this is not a security issue.

Also note that 3.9 will go to end of life by the end of the year (which means it will receive no further updates from us from that point forward, not even security updates)

[sergey-miryanov]

@gabe-sherman It would be wonderful if you point in following issues that you are fuzzing.