To-do list for 2.14.0

[DanielNoord]

TODO:

• Remove set_config_directly #6123
• Remove python 3.6 specific code and stop supporting python 3.6
  • Require python 3.7.2+ #5921
  • Cleanup after we drop support for Python 3.6 #4683
  • Update tbump requirement from ~=6.6.0 to ~=6.7.0 #5600
  • To-do list for 2.14.0 #5513 (comment)
  • Update coverage requirement from ~=6.2 to ~=6.3 #5751
• Add classifications to TODOs across codebase #6320
• Release branch 2.14 beta #6527
• Fix any encountered issues during the beta
• Update all TODO: 2.14 #6573
• Release branch 2.13.9 #6563
• Reorganize the documentation table of content #6589
• Release 2.14.0, ideally after May 20th

Potentially:

• Update astroid dependency to 2.12 if we manage to release it before 2.14 is ready. Related: Require Python 3.7.2 astroid#1542

[DanielNoord]

I'm adding to this that https://bugs.python.org/issue33053 seems to have been fixed in 3.7. We mention that issue in here:
https://github.com/PyCQA/pylint/blob/019794b808271d45f86a7014e9c91cb04458a47a/pylint/__init__.py#L56

We need to update the docstring to reflect whatever the situation in 3.7+ is.

[cdce8p]

We could also consider if we want to require 3.7.2 instead of 3.7.0. That would prevent any upcoming issues with NoReturn. Just to be clear though, the current code works with 3.7.0 just fine.
For reference #5304.

[DanielNoord]

I'm adding to this that https://bugs.python.org/issue33053 seems to have been fixed in 3.7. We mention that issue in here:

https://github.com/PyCQA/pylint/blob/019794b808271d45f86a7014e9c91cb04458a47a/pylint/__init__.py#L56

We need to update the docstring to reflect whatever the situation in 3.7+ is.

I re-read that issue and there is no action to be taken here. We handle this correctly. Ticking off.

[DanielNoord]

EDIT: Never mind. This discussion came up again in CPython and pylint was actually used as an example of how this can cause unwanted behaviour. Let's keep as is and try to improve our namespace package recognition.

@cdce8p @Pierre-Sassoulas @jacobtylerwalls

I have been trying to warp my head around #2862 as the issue has a fairly easily reproducible example and should allow narrowing down the namespace issue.

One thing I was wondering is whether this function is truly necessary:
https://github.com/PyCQA/pylint/blob/019794b808271d45f86a7014e9c91cb04458a47a/pylint/__init__.py#L49-L67

I'm asking this because with the following diff it shows that we do not recover path correctly:

diff --git a/pylint/lint/utils.py b/pylint/lint/utils.py
index 792c47f2d..8c6fc427d 100644
--- a/pylint/lint/utils.py
+++ b/pylint/lint/utils.py
@@ -117,6 +117,8 @@ def fix_import_path(args: Sequence[str]) -> Iterator[None]:
     `sys.path` is reset to its original value upon exiting this context.
     """
     original = _patch_sys_path(args)
+    for i in sys.path:
+        print(i)
     try:
         yield
     finally:

❯ python -m tools.foo.foo
/Users/daniel/DocumentenLaptop/Programming/Github
/Users/daniel/.pyenv/versions/3.10.0/lib/python310.zip
/Users/daniel/.pyenv/versions/3.10.0/lib/python3.10
/Users/daniel/.pyenv/versions/3.10.0/lib/python3.10/lib-dynload
/Users/daniel/DocumentenLaptop/Programming/Github/venv/lib/python3.10/site-packages
/Users/daniel/DocumentenLaptop/Programming/Github/astroid
/Users/daniel/DocumentenLaptop/Programming/Github/pylint
❯ python -m pylint ./tools/foo/foo.py
/Users/daniel/DocumentenLaptop/Programming/Github/tools/foo
/Users/daniel/.pyenv/versions/3.10.0/lib/python310.zip
/Users/daniel/.pyenv/versions/3.10.0/lib/python3.10
/Users/daniel/.pyenv/versions/3.10.0/lib/python3.10/lib-dynload
/Users/daniel/DocumentenLaptop/Programming/Github/venv/lib/python3.10/site-packages
/Users/daniel/DocumentenLaptop/Programming/Github/astroid
/Users/daniel/DocumentenLaptop/Programming/Github/pylint

Before trying to fix namespace packages I wonder if the sys.path mangling we do is completely necessary. Note that in the CPython issue the conclusion is that Python is intended for user computers and that it does not guarantee that no malicious code will be ran.
It seems easier to special case stdlib or pylint's own modules instead of changing sys.path which then makes it almost impossible to figure out namespace packages. If we believe this should remain I'll try and see if there is another way to do those, although after poking around for a couple of hours I'm not too hopeful.

Edit: Original issue is #3386.

[DanielNoord]

One final issue here is related to astroid. We need the ComprehensionScope PR to land to fix the coverage drop indicated by the last TODO: 2.14.
I think we can close this as long as we remember to remove those whenever we finally release astroid with that PR in it.

[DanielNoord]

I'm going to move the TODO's that are scheduled for 2.14 to something else as the ComprehensionScope PR has been closed after discussion with @cdce8p.

That means we only need to wait for the beta-period to run out before we can release 2.14. Since many dependency updates seem to take place on Monday ideally I would like to wait at least until May 20th. I'm going to update the OP, but please change the deadline if anybody wants to!

In the meantime I think we can probably focus on getting astroid 2.12 out with pylint-dev/astroid#1542 merged. Seeing as that we managed to release various patch versions while main was on 3.7.2+ while maintenance/2.13.x was not in pylint I think not much is blocking us there.
@jacobtylerwalls do you also want to handle the py-version and related from __future__ import annotations PRs? I can help review those, but feel free to stick to pylint-dev/astroid#1542.

[jacobtylerwalls]

@jacobtylerwalls do you also want to handle the py-version and related from future import annotations PRs? I can help review those, but feel free to stick to pylint-dev/astroid#1542.

No concrete plans to get to it ATM, but if I can pitch in at some point I will!

[cdce8p]

@jacobtylerwalls do you also want to handle the py-version and related from future import annotations PRs? I can help review those, but feel free to stick to PyCQA/astroid#1542.

No concrete plans to get to it ATM, but if I can pitch in at some point I will!

I've started updating the astroid typing following the bump to 3.7.2.
The first PR: pylint-dev/astroid#1555

Edit: Astroid typing update ✅