python language server uses too much memory

[DonJayamanne]

@lisnb commented on Fri May 31 2019

Issue Type: Bug

I'm using this amazing Remote-SSH extension, but the python language server on the remote uses a lot of memory up to 120G+, I try to unistall this extension or kill -9 the process, but it's not working. It seems this process starts itself once killed.

Extension version: 2019.5.17517
VS Code version: Code - Insiders 1.35.0-insider (e9c50663154c369a06355ce752b447af5b580dc3, 2019-05-30T05:18:06.502Z)
OS version: Darwin x64 18.2.0
Remote OS version: Linux x64 4.4.0-33.bm.1-amd64
Remote OS version: Linux x64 4.4.0-33.bm.1-amd64

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz (4 x 2500)
GPU Status	2d_canvas: enabled checker_imaging: disabled_off flash_3d: enabled flash_stage3d: enabled flash_stage3d_baseline: enabled gpu_compositing: enabled multiple_raster_threads: enabled_on native_gpu_memory_buffers: enabled rasterization: enabled surface_synchronization: enabled_on video_decode: enabled webgl: enabled webgl2: enabled
Load (avg)	2, 2, 2
Memory (System)	16.00GB (0.49GB free)
Process Argv
Screen Reader	no
VM	0%

Item	Value
Remote	SSH: dev.29
OS	Linux x64 4.4.0-33.bm.1-amd64
CPUs	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz (48 x 2301)
Memory (System)	251.31GB (70.78GB free)
VM	0%

Item	Value
Remote	SSH: dev.29
OS	Linux x64 4.4.0-33.bm.1-amd64
CPUs	Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz (48 x 2301)
Memory (System)	251.31GB (70.78GB free)
VM	0%

---

@lisnb commented on Fri May 31 2019

I noticed that this process was located in ~/.vscode-remote directory but as the remote-ssh extension keeps updating, the vscode is now under ~/.vscode-server-insider directory

[MikhailArkhipov]

@lisnb, please provide version of the LS (it is displayed at startup) and the list of libraries/packages you are importing.

[CMLL]

I ran into this issue today after a long time of the LS behaving pretty ok compared with the previous versions.

LS Version 0.2.98.0

Started VSCode and let it sit on a project that I open daily. Took no action on it, just let it sit. Memory started being consumed without stopping until the machine would become unresponsive, the language server process would be killed and restarted again and the process would repeat itself, keep consuming memory until it crashes again.

I noted also the the background analysis message would be stuck on the same number without decreasing during this whole time. Also noticed that an exception of what appeared to be the Intellicode extension caused a restart of the analysis which then would fire the whole memory consuming process.

I uninstalled the Intellicode extension to see if removing that would help but I see still that the analysis message keeps stuck in a high number, even though this time around there was no exception on the logs. However memory still kept growing, I killed it before it got to the point of crashing my machine.

I'm not sure if it has to do with anything but one thing I did different today was installing the new remote development extensions that got released with vscode 1.35, but I haven't gotten around to using them yet so I'm not sure if that's related or not.

Not sure if this is accurate but I enabled Trace logging into the extension and some of times when logs would stop at this point and never log anything else. Not sure if something in there the cause of the background analysis never finishing.

[MikhailArkhipov]

@CMLL - I can see in the log that modules are getting reloaded. This means LS decided that packages changed. LS needs to distinguish libraries from user code. @jakebailey worked in this.

Do you happen to have search paths pointing to your code so perhaps LS is confusing file save with package installation?

[CMLL]

Yeah, I do.
I have this in my workspace settings...

    "python.autoComplete.extraPaths": [
        "/home/cllamach/Panopta/classic/src/",
    ],

It points to the app that I am working on currently. For other reasons that path is also part of my local PYTHONPATH so the apps I'm running can detect inner model requirements, but this is separate from all the library dependencies which live in a .virtualenvs folder at $HOME.

[jakebailey]

That's outside of the folder you have open, correct? We currently only treat code inside the workspace (that isn't an interpreter file, for users who put their venvs in the workspace) as user code, so if it's outside, it'll be treated as library code, which I believe will also mean that it will cause a reload when it changes.

[CMLL]

Actually it isn't. The current workspace I have open is /home/cllamach/Panopta/classic. One question, where are the Configuration search paths being determined from? Currently is showing this on LS start.

Which are the values I had set on my PYTHONPATH variable in the shell, but I removed those from there and they still show up on the LS.

[jakebailey]

Do you see your log spammed with the "reloading modules" message, or just the two times shown in the screenshot? Are you also setting PYTHONPATH somewhere?

As an aside, you can provide paths relative to the workspace root in that option, so you can specify "./src" and not have to hardcode the entire path on your filesystem.

[jakebailey]

Configuration search paths are those given by PYTHONPATH and extraPaths, merged together into a single list (by the python extension before we start). After that logging, we go and classify what's in the workspace and what isn't to see what's user code (and I've been planning on moving those prints somewhere later to get a real picture).

[CMLL]

They are not being spammed constantly, however the background analysis still doesn't complete, or at least the message does not go away.

If configuration search path work that way, then I don't really need to set the extraPaths variables, but I've been trying to have the LS start without those 2 paths to discard something there and no luck. I must have a PYTHONPATH set somewhere.

Update:
Was able to start the LS without 2 configuration search paths included and the analysis still seems to stuck somewhere. Memory usage seems to keep increasing steadily, didn't let it crash my machine but it def looked like that was going to be the end result.

[jakebailey]

With your setup (a src directory below the workspace root), you'll have to specify ./src as a search path at least somewhere, otherwise we won't be able to resolve your imports. If your output isn't being spammed with the reloading, then it's probably not a file watch or directory classification problem, but is something unrelated. Is this project somewhere we can test it?

[CMLL]

Sadly no, is a private project for the company I work for. Removing the configuration search paths doesn't to fix the issue so it leads me to believe it's not a inner workspace code issue, but it could be a virtualenv issue with one of the installed libraries, which is pretty big list.

Which makes sense, other projects with different virtualenvs don't seem affected by the same issue, at least in some preliminary testing I did.

[CMLL]

Seems like is indeed some thirdparty library in the virtualenv folder that's causing the issue, moving the workspace python interpreter to the system python lets the analysis complete fine.