-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Script to run all checks locally #8798
Script to run all checks locally #8798
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! This looks like a great start.
I have a few questions, a few suggestions, and a few nits ;)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
…sam/typeshed into run-all-checks-single-script
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
…sam/typeshed into run-all-checks-single-script
…sam/typeshed into run-all-checks-single-script
Here's the output currently if I try running `python scripts/runtests.py redis`.(.venv) C:\Users\alexw\coding\typeshed>python scripts/runtests.py stubs/redis
Looks good! ✨ 🍰 ✨
50 files left unchanged.
All done! ✨ 🍰 ✨
50 files left unchanged.
error finding npx; is Node.js installed?
*** Testing Python 3.11 on win32
Testing third-party packages...
Running mypy --python-version 3.11 --show-traceback --warn-incomplete-stub --show-error-codes --no-error-summary --platform win32 --no-site-packages --custom-typeshed-dir C:\Users\alexw\coding\typeshed --no-implicit-optional --disallow-untyped-decorators --disallow-any-generics --strict-equality --enable-error-code ignore-without-code --config-file /tmp/...
testing redis (50 files)... success
*** Testing Python 3.10 on win32
Testing third-party packages...
Running mypy --python-version 3.10 --show-traceback --warn-incomplete-stub --show-error-codes --no-error-summary --platform win32 --no-site-packages --custom-typeshed-dir C:\Users\alexw\coding\typeshed --no-implicit-optional --disallow-untyped-decorators --disallow-any-generics --strict-equality --enable-error-code ignore-without-code --config-file /tmp/...
testing redis (50 files)... success
*** Testing Python 3.9 on win32
Testing third-party packages...
Running mypy --python-version 3.9 --show-traceback --warn-incomplete-stub --show-error-codes --no-error-summary --platform win32 --no-site-packages --custom-typeshed-dir C:\Users\alexw\coding\typeshed --no-implicit-optional --disallow-untyped-decorators --disallow-any-generics --strict-equality --enable-error-code ignore-without-code --config-file /tmp/...
testing redis (50 files)... success
*** Testing Python 3.8 on win32
Testing third-party packages...
Running mypy --python-version 3.8 --show-traceback --warn-incomplete-stub --show-error-codes --no-error-summary --platform win32 --no-site-packages --custom-typeshed-dir C:\Users\alexw\coding\typeshed --no-implicit-optional --disallow-untyped-decorators --disallow-any-generics --strict-equality --enable-error-code ignore-without-code --config-file /tmp/...
testing redis (50 files)... success
*** Testing Python 3.7 on win32
Testing third-party packages...
Running mypy --python-version 3.7 --show-traceback --warn-incomplete-stub --show-error-codes --no-error-summary --platform win32 --no-site-packages --custom-typeshed-dir C:\Users\alexw\coding\typeshed --no-implicit-optional --disallow-untyped-decorators --disallow-any-generics --strict-equality --enable-error-code ignore-without-code --config-file /tmp/...
testing redis (50 files)... success
--- success, 250 files checked ---
redis... success
Skipping pytype on Windows. You can run the test with WSL.
usage: regr_test.py [-h] [--all] [--platform [{linux,darwin,win32} ...]] [-p [{3.11,3.10,3.9,3.8,3.7} ...]] [packages_to_test ...]
regr_test.py: error: argument packages_to_test: No test cases found for 'redis'!
One or more tests failed. See above for details.
flake8: Success
Check consistent: Success
Check new syntax: Success
pyright: Failed
mypy: Success
stubtest: Success
pytype: Skipped
Regression test: Success Some notes:
|
…all-checks-single-script
(Nit): we currently get a bit of an ugly traceback if the script is interrupted halfway through: (.venv) C:\Users\alexw\coding\typeshed>python scripts/runtests.py stubs/redis
Running pycln...
Looks good! ✨ 🍰 ✨
50 files left unchanged.
Traceback (most recent call last):
File "C:\Users\alexw\coding\typeshed\scripts\runtests.py", line 55, in <module>
subprocess.run([sys.executable, "-m", "pycln", path, "--all"])
File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 503, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1144, in communicate
self.wait()
File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1207, in wait
return self._wait(timeout=timeout)
File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1488, in _wait
result = _winapi.WaitForSingleObject(self._handle,
KeyboardInterrupt
^C It might be nice to wrap the whole Lines 179 to 185 in 380022c
|
…all-checks-single-script
A traceback can still appear, but it's clearer why. Unless maybe we start piping the output of all tests, checking for a KeyboardInterrupt and then printing the output. I don't think that's really worth it. |
Agreed, it looks good now! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some very minor suggestions about some of the terminal output
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thank you! Just one last comment, but other than that -- I've run out of things to say!
I'll leave this open for a day or two in case any other maintainers want to take a look, but this looks great to me :)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Closes #8686
There's still an issue that needs to be fixed with the mypy test (see #8797), and I would like to be able to seamlessly run the pytype test on windows, if that's even feasible.
Summary output:

Please let me know how to better word the "next steps" in
create_baseline_stubs
.Notes of things to do in follow-up PR(s):