Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

Unpacking typing Tuples and Lists correctly #1553

Merged
merged 86 commits into from
Sep 16, 2019
Merged

Unpacking typing Tuples and Lists correctly #1553

merged 86 commits into from
Sep 16, 2019

Conversation

MikhailArkhipov
Copy link

#1498 BY @CTrando via db branch

Fixes #1457
Fixes #1494
Fixes #411
Fixes #1382
Fixes #1293

Mikhail Arkhipov and others added 30 commits June 20, 2019 09:16
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (#1227)

* Fix addBrackets setting (#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (#1246)

* Trying to augment argument set with Evaluator and Expression context (#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (#1285)

* Multiple analysis fixes (#1297)

Multiple analysis fixes.
- Fixes #1151: Reliable way to determine final analysis of a module
- Fixes #1201: NRE in SymbolCollector.AddProperty
- Fixes #1228: NRE in TryFindMissingDependencies
- Fixes #1294: Handle valid case for reloading AST
- Fixes #1295: FindReferences doesn't work on package init members
- Fixes #1296: DependencyResolver graph misses intermediate imports
- Part of the fix for #1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (#1405)

* Remove all references to dropped modules and force GC on reload (#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback
…y! (#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast
MikhailArkhipov and others added 27 commits September 11, 2019 09:57
Fix import model restore
* Preliminary zip file

* Refactoring and adding .egg support

* refactoring

* More refactoring

* removing irrelevant classes

* Preliminary working in memory

* More things to make it work

* Getting rid of comment

* Refactoring

* More refactoring

* more changes

* refactoring

* Removing unneeded changes

* adding back

* PR feedback

* Test fix
* Handling unpacking with typing structures

* Removing changes to value enumerator

* Test fix, formatting

* Refactors

* Handling nested structures

* Handles non typing lists and tuples

* Giving up on fixing root problem, adding hack

* test fix

* Reverting change to test and fixing test

* Update syntax

* Checking #1494

* Handling #411 as well

* multiple assignment + unpacking fixes #1293

* Handling more cases with unpacking nested tuples

* merge fixes

* No default param for arg set, force people to use something
@MikhailArkhipov MikhailArkhipov merged commit 57a9a43 into master Sep 16, 2019
jakebailey pushed a commit to jakebailey/python-language-server that referenced this pull request Nov 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.