Skip to content

Don't keep AST in memory, lazy load content on demand #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

felixfbecker
Copy link
Owner

@felixfbecker felixfbecker commented Oct 10, 2016

Extension of #60

Now only a FQN => PhpDocument map is kept at the project level.
Upon a definition request the content/AST of the document is lazily loaded from disk.

Indexing is still slow but RAM usage is now as low as <100MB even for magento (for some reason though VS Code crashed after ~20,000 files).

I moved all LSP-related logic like findSymbols out of Project and PhpDocument into the method handlers. Project and PhpDocument in turn provide methods like getDefinition or getDefinitionDocument.

workspace/symbol now also searches over the FQN, which is a nice side-effect.
The downside though is that workspace/symbol now needs to reparse every document matching the query. So in the case that your query is not very specific, it will take forever in large projects and bring your RAM usage up to the sky.

Test with Symfony (7100 files):

All PHP files parsed in 259 seconds. 52 MiB allocated.

@codecov-io
Copy link

codecov-io commented Oct 10, 2016

Current coverage is 90.33% (diff: 93.10%)

Merging #61 into master will increase coverage by 0.41%

@@             master        #61   diff @@
==========================================
  Files            23         24     +1   
  Lines           496        507    +11   
  Methods          70         73     +3   
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits            446        458    +12   
+ Misses           50         49     -1   
  Partials          0          0          
Diff Coverage File Path
•••••••• 89% src/PhpDocument.php
•••••••••• 100% src/Server/TextDocument.php
•••••••••• 100% src/Server/Workspace.php
•••••••••• 100% src/Project.php
•••••••••• 100% src/LanguageServer.php
•••••••••• 100% src/Protocol/SymbolInformation.php

Powered by Codecov. Last update e75c159...58fb1b9

@mniewrzal
Copy link
Contributor

With this PR I have reasonable indexing times:
magento

[Info  - 9:57:14 PM] Parsing file 24273/24273: dev/tests/functional/tests/app/Magento/Email/Test/Constraint/AssertEmailTemplateSuccessSaveMessage.php.
[Info  - 9:57:14 PM] All PHP files parsed in 119 seconds. 575 MiB allocated.

symfony

[Info  - 9:58:36 PM] Parsing file 7099/7100: vendor/zendframework/zend-code/test/Generic/Prototype/PrototypeClassFactoryTest.php.
[Info  - 9:58:36 PM] Parsing file 7100/7100: .github/build-packages.php.
[Info  - 9:58:36 PM] All PHP files parsed in 27 seconds. 50 MiB allocated.

To be clear I'm testing with XDebug disabled.

@felixfbecker felixfbecker deleted the lazy-load branch October 11, 2016 12:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants