You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The "pythoneval" tests are by far the biggest contributor to the time it takes to run our test suite. For each of 171 sample programs, we type-check them and then run them through the Python interpreter.
From discussion in #1668, running the programs is no longer important for our testing. So cut it out. This will mean
cutting out that step in testpythoneval.py,
cutting out the program output (as opposed to the type-checker output) from all the test cases, and
cutting out entirely any cases that no longer seem to have any meaningful content when they're only for the type-checker.
Some of the test cases may exercise features of typing in interesting ways -- any of those should be moved to the typing repo's own test suite if they aren't covered there already.
After this is done, we can probably get a big further speedup by folding the remaining test cases into just more cases for testcheck.py, so that we're not forking a mypy subprocess for each one. Will make a separate issue for that.
The text was updated successfully, but these errors were encountered:
This replaces the old subprocess based method. In addition, the test cases
are no longer run, in order to reduce test time. This has led to a 10-20%
speedup in pytest. This might help with python#3895, as it removes the
subprocesses.
Fixespython#1671
I just tested completely removing this and it doesn't seem to actually be that big of a runtime hit. WIth Guido's points about checking real code: #4025 (comment), I think we can just close this, the majority of the runtime seems to be in type checking the files.
The "pythoneval" tests are by far the biggest contributor to the time it takes to run our test suite. For each of 171 sample programs, we type-check them and then run them through the Python interpreter.
From discussion in #1668, running the programs is no longer important for our testing. So cut it out. This will mean
testpythoneval.py
,Some of the test cases may exercise features of
typing
in interesting ways -- any of those should be moved to thetyping
repo's own test suite if they aren't covered there already.After this is done, we can probably get a big further speedup by folding the remaining test cases into just more cases for
testcheck.py
, so that we're not forking a mypy subprocess for each one. Will make a separate issue for that.The text was updated successfully, but these errors were encountered: