Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

AST analyzer and assemblies split #465

Merged
merged 180 commits into from
Jan 18, 2019
Merged

AST analyzer and assemblies split #465

merged 180 commits into from
Jan 18, 2019

Conversation

MikhailArkhipov
Copy link

@MikhailArkhipov MikhailArkhipov commented Dec 7, 2018

Fixes #426
Fixes #408
Fixes #473

Base of of #406

Microsoft.Python.Core - common objects usable in many places
Microsoft.Python.Parsing - Python parser and AST
Microsoft.Python.Analysis.Engine - current DDG analysis
Microsoft.Python.Analysis.Core - dependency resolution and other code common to DDG and AST.
Microsoft.Python.Analysis - AST analysis

  • No logic changes in DDG analysis apart from minor changes in references due to parser and core utilities relocation.

AST analysis:

  • New test assembly with 92 tests
  • Updated fluent assertions in the new test assembly
  • Asynchronous document loading and analysis
  • Asynchronous imports
  • Asynchronous AST walk
  • Removed lazy types
  • Removed 'multiple types' constructs
  • Stub is imported first
  • Functions are not analyzed if stub equivalent is present. Instead, overload instance from the stub is used.
  • Partial transfer of DDG functionality (Transfer type determination parts from DDG walker to AST walker #407).

@jakebailey
Copy link
Member

Running the tests before approving this:

  • ListCallCtor fails: Expected Type.TypeId to be Int, but found Unknown.
  • Many of the parsing tests fail
    • MutateStdLib gives things like: Assert.AreEqual failed. Expected:<5>. Actual:<4>.
    • Most roundtrip tests fail, like: Assert.AreEqual failed. Expected:<*>. Actual:< >. Characters differ at 21, got , expected *

@jakebailey
Copy link
Member

Alright, the tests seem to pass now, other than the FullStdLibAnaconda3 scrape test timing out. Building/running the old language server works as well.

Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess now is better than never. A lot of the in-progress items will probably need to be closed after this is merged.

@MikhailArkhipov MikhailArkhipov merged commit 7ec9561 into microsoft:master Jan 18, 2019
@MikhailArkhipov MikhailArkhipov deleted the analysis5 branch January 18, 2019 19:47
jakebailey pushed a commit to jakebailey/python-language-server that referenced this pull request Nov 1, 2019
* Types hierarchy cleanup

* Test fixes, part I

* Restore IPythonBoundFunction

* Fix function instance types

* More fixes

* Fix overload handling

* Fix properties

* Fix type factory

* Remove class built-in type

* Simplify builtins, remove obsolete method

* Fix typeshed merge

* Allow declaring module to be null

* Correct assertion

* Simplify functions/methods

* Fix stub merge tests

* Fix tests

* Baselines

* Null ref

* Build break

* Overrides signatures

* Fix typeinfo tests

* PR feedback

* Infra to separate assembly

* Parser to own assembly

* Parser tests to own assembly

* Unify to IndexSpan

* Better handle overloads

* Reorg, part I

* Part II

* Part III

* Part 4

* Part 5

* Build breaks

* Baselines

* PR fix

* Merge build issues

* Part 6

* Part 7

* Buildable

* PR feedback

* Merge conflict

* Fix microsoft#446

* Fix microsoft#446

* Part 8

* Part 9

* Buildable

* Part 10

* Part 11

* Part 12

* Buildable

* Part 14

* First passing test

* Simplify configuration

* Style

* Fix test and move code to folders

* Builtins import

* Fluents

* Add search path

* Import analysis, part I

* Simplify builtins handling

* Remove IMember

* Handle import specific

* More tests

* Add typeshed

* Renames

* Make sure lazy modules are loaded

* Renames

* Move/rename

* Rework importing

* Derivation rework

* Part 2

* Part 3

* Buildable

* Module members

* Async walk

* Imports test pass

* Remove lazy types

* Fix from import

* Stubs

* Double overloads

* Fix datetime test

* Couple more tests + fluents

* Few more tests

* Additionl test + union type

* Built-in scrape tests

* Full stdlib scrape test

* Complete async AST walker

* Conditional defines test + variable loc cleanup

* More stub tests
Fix stub loading for packages (port from DDG)
Split walker into multiple files

* Add some (broken mostly) tests from DDG

* Move document tests

* Function arg eval, part I

* Instance/factory

* Builds

* Test fixes

* Fix static and instance call eval

* More tests

* More ported tests

* Specialize builtin functions

* Make walkers common and handle nested functions

* Moar tests

* Parser fixes + more tests

* Handle negative numbers

* Fix null ref

* Basic list support

* Few more list tests

* Basic iterators

* Support __iter__

* Iterators

* Fix couple of tests

* Add decorator test

* Generics, part I

* Generics, part 2

* Generics, part 3

* Basic TypeVar test

* Typings, part 4

* Fix test

* Generics, part 6

* Generics, part 7

* More tests (failing)

* Forward ref fixes

* Reorg

* Improve symbol resolution + test fixes

* Test fixes

* Dictionary, part I

* Part 11

* Fix test

* Tests

* Tests

* More dict work

* List ctor

* Skip some tests for now

* Fix iterators

* Tuple slicing

* Polish type comparo in return types

* Add Mapping and tests

* Add Iterable

* Fix typo

* Add Iterator[T] + test

* Simplify typing types

* Class reduction

* Fix tests

* Test fix

* Handle 'with' statement

* Handle try-except

* Class method inheritance + NewType

* Container types

* Containers test

* Tests

* Handle generic type alias

* Named tuple

* Global/non-local

* Handle tuples in for
Handle custom iterators

* Basic generator

* Any/AnyStr

* Test fixes

* Type/Optional/etc handling

* Proper doc population

* Tests + range

* Argument match

* Basic argset and diagnostics

* Argset tests

* Exclude WIP

* Exclude WIP

* Arg eval

* Arg match, part 2

* Tests and generic arg comparisons

* Function eval with arguments

* Baselines

* Fix test

* Undo AST formatting change and update baseline

* Fix list ctor argument unpacking
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants