Run pylance tests in CI

.github/workflows/tests.yml

@@ -85,6 +85,14 @@ jobs:
       - uses: actions/setup-python@v2
       - run: ./tests/mypy_test_suite.py
 
+  pyright:
+    name: Run the pyright test suite
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+      - run: ./tests/pyright_test.py
+
   stubtest:
     name: Check stdlib with stubtest
     runs-on: ${{ matrix.os }}

README.md

@@ -108,6 +108,8 @@ There are several tests:
 tests typeshed with [mypy](https://github.com/python/mypy/)
 - `tests/pytype_test.py` tests typeshed with
 [pytype](https://github.com/google/pytype/).
+- `tests/pyright_test.py` tests typeshed with
+[pyright](https://github.com/microsoft/pyright).
 - `tests/mypy_self_check.py` checks mypy's code base using this version of
 typeshed.
 - `tests/mypy_test_suite.py` runs a subset of mypy's test suite using this version of
@@ -121,14 +123,17 @@ consistent with each other.
 
 Run:
 ```
-$ python3.6 -m venv .venv3
+$ python3 -m venv .venv3
 $ source .venv3/bin/activate
 (.venv3)$ pip install -U pip
 (.venv3)$ pip install -r requirements-tests-py3.txt
 ```
 This will install mypy (you need the latest master branch from GitHub),
 typed-ast, flake8 (and plugins), pytype, black and isort.
 
+If you want to run the pyright tests, you need to have
+[Node.js](https://nodejs.org/) installed.
+
 ### mypy_test.py
 
 This test requires Python 3.6 or higher; Python 3.6.1 or higher is recommended.
@@ -160,6 +165,10 @@ Run using: `(.venv3)$ python3 tests/pytype_test.py`
 
 This test works similarly to `mypy_test.py`, except it uses `pytype`.
 
+### pyright\_test.py
+
+This test requires Node.js to be installed.
+
 ### mypy_self_check.py
 
 This test requires Python 3.6 or higher; Python 3.6.1 or higher is recommended.

tests/pyright_test.py

@@ -0,0 +1,52 @@
+#!/usr/bin/env python3
+
+import os
+import subprocess
+import sys
+from pathlib import Path
+from typing import Iterable, List
+
+
+_WELL_KNOWN_FILE = Path("tests", "pyright_test.py")
+_PYRIGHT_COMMAND = ["npx", "-p", "[email protected]", "pyright"]
+
+
+def main() -> None:
+    assert_npm_is_installed()
+    files = find_stubs()
+    ret = run_pyright(files)
+    sys.exit(ret)
+
+
+def assert_npm_is_installed() -> None:
+    if not _WELL_KNOWN_FILE.exists():
+        print("pyright_test.py must be run from the typeshed root directory", file=sys.stderr)
+        sys.exit(1)
+    try:
+        subprocess.run(["npx", "--version"])
+    except OSError:
+        print("error running npx; is Node.js installed?", file=sys.stderr)
+        sys.exit(1)
+
+
+def find_stubs() -> List[str]:
+    files: List[str] = []
+    for top in ["stdlib", "stubs"]:
+        for dirpath, dirnames, filenames in os.walk(top):
+            for fn in filenames:
+                if fn.endswith(".pyi"):
+                    files.append(os.path.join(dirpath, fn))
+            try:
+                dirnames.remove("@python2")
+            except ValueError:
+                pass
+    return files
+
+
+def run_pyright(files: Iterable[str]) -> int:
+    full_args = _PYRIGHT_COMMAND + list(files)
+    return subprocess.run(full_args).returncode
+
+
+if __name__ == "__main__":
+    main()