Extension process using too much memory in the filename cache #4714
Comments
bobbrow
changed the title
|
@Gargony replied: @sean-mcmanus may be this help |
|
@sean-mcmanus replied: @Gargony We fixed a high hitting crash with 0.26.0.1 involving "check_and_compile"...if you have 0.26.1 or later, then you could be hitting a less commonly hitting crash with that method. We might need an isolated/sample repro to fix that. Is the memory usage related to the crash or are they separate issues? What causes the memory to get to 4 GB? Just opening a file? Or does it accumulate as the cursor is moved around the file? Or only after edits? |
|
@bobbrow I got crash with memory up to 4 GB on Microsoft.VSCode.CPP.Extension.exe (0.26.2.2) |
|
Hi @Gargony . Do you have a huge number of files in your project? It looks like you're running out of memory (on a 32-bit system?) while building a table of header/source relationships from data in the tag parser database. |
|
Hi @Colengms . I'm not sure about border huge and less files ))) We have a mobile application about ~1K+ cpp files. boost, stl, templates like this https://github.com/DevUtilsNet/linqcpp/blob/master/include/linqcpp/linqcpp.h and etc. Nothing special, usual project ))) Please tell me what information I can provide to you? I can provide compile_commands.json (3Mb+), build.ninja (4Mb+) or somphing like this. |
|
Does the bug re-appear after doing the Reset IntelliSense Database, i.e. does it seem like the database was in a bad state due to a previously fixed issue? |
|
@sean-mcmanus Yes, the bug re-appear after I doing the Reset IntelliSense Database. |
|
@sean-mcmanus after clear "ipch" the process Microsoft.VSCode.CPP.Extension.exe works and after about ~3h is began crush ("ipch" ~830Mb) |
|
Hi @Gargony . The specific allocation that failed is an association between a header and a file that includes it. This information is used to select an appropriate source file when creating an IntelliSense Translation Unit, as a header file is opened. It's possible that a very large number of #include statements (not necessarily files) could lead to excessive memory usage. Running out of memory here is surprising. It sounds like we may need to rethink how this is implemented. Do you consistently see the same stack (within populate_filename_cache) ? If you see any other stacks associated with the issue, they may provide some clues. In the meanwhile, I will see what we can do to reduce the overhead involved in includer/includee associations. |
|
Hi @Colengms the stack is the same |
|
@Gargony I just see a "bad_alloc" with no other call stack. Are you able to attach a debugger and set "symbolSearchPath" to "https://msdl.microsoft.com/download/symbols" to get a better call stack? We'll try to release 0.26.3-insiders Monday which may fix the issue via using less memory. |
|
Ignore my last post -- we got the call stack from the dmp. To double check, after you do a Reset IntelliSense Database, how long does it take to finish parsing? 3 hours? After the database is built, how long does it take for the crash to occur? How fast do you see the memory usage increase? Over a matter of seconds or minutes or hours? We'll try to add more logging to our 0.26.3-insiders build (tentatively for Monday). |
|
Hi @sean-mcmanus , here https://1drv.ms/u/s!Ai8Z7kEn6DQTmKwCzpBTWFPzZ9cBig?e=Skb8qC you can see how memory is grows. After Reset IntelliSense Database the vscode-cpptools take a few seconds and all works fine until next apisode. Microsoft.VSCode.CPP.Extension.exe (0.26.2.2) |
|
https://github.com/microsoft/vscode-cpptools/releases/tag/0.26.3-insiders has a fixed that reduced the memory usage -- is anything better with that? We added more logging with the hidden value of "7", but for some strange reason I'm not seeing that logging (might need to debug it tomorrow). But 0.26.3-insiders has a known IntelliSense crash regression (#4752) if you happen to hit that. |
|
I see the "Files in filename cache" logging now -- the cache has to be regenerated. Let us know if you are able to get any "filename cache" logging info. UPDATE: FYI, we've unpublished 0.26.3-insiders until we can get the crash regression fixed. |
|
@sean-mcmanus Sorry but I don't see any helpful info in C/C++ output. This output taken when the Microsoft.VSCode.CPP.Extension.exe (0.26.3.0) in bug state (out of memory and restarting) |
|
Your loggingLevel needs to be set to the hidden value of "7" to see the new "filename cache" logging we added in Insiders (we unpublished 0.26.3-insiders though due to an unrelated crash regression, that we're hoping to have a fix for today). It sounds like you're saying the memory usage improvements didn't help? |
Yes, didn't help. |
|
Here output with log level 7 https://onedrive.live.com/?authkey=%21AM6QU1hT82fXAYo&id=1334E82741EE192F%21398850&cid=1334E82741EE192F (see Output.txt)
|
|
What do you mean "the output contains not all lines"? Why are you not able to add more lines to the log? We're looking for output around that has text "filename cache" because our hypothesis is that that's the new code that is causing the memory usage issue. Do you see any of that in the logs? The log doesn't show anything interesting other than the initial file scan. How does the memory usage relate to those logs? The code that potentially is causing the problem iterates through the files in the database and forms links between source files and header files that are incuded by the source files. Can you think of anything special with your workspace that might cause this code to loop infinitely? We have not been able to replicate this condition yet with our testing. |
|
There were no crashes after several days of observation. |
|
Hi @Gargony . How many files do you have in this project/folder? Can you tell if memory usage grew slowly over time, or spiked suddenly? The stack you caught for this (related to populating a cache of header/source relationships) is only invoked as a header is opened, and only if there is not another file already open that includes that header (directly or indirectly). The first time this happens, or if files have been added/removed/moved since the last time, we build a table of header/source relationships, to determine a good source file to use instead of the header. That table should not be particularly large. With 50K files, I'm seeing memory usage of ~5M. Memory usage would spike suddenly. If you are not using a huge number of files, and memory usage grew steadily instead of spiking, the root cause could be a leak elsewhere, and manifesting there due to the spike.. If you can catch a full heap dump for the out of memory crash, that could be very useful. :) |
bobbrow
changed the title
|
@Gargony We believe we have a fix for this in 0.26.3-insiders3 . If you are able to repro again, could you confirm? |
Colengms
added
the
fixed
|
@Colengms thx, I'll check it. |
|
We shipped 0.26.3 -- we can reopen this if it still repros: https://github.com/microsoft/vscode-cpptools/releases/tag/0.26.3 . |
@sean-mcmanus Yes, it's happens during code write, I get (cpptools:.... )"Connection to server got closed. Server will not be restarted", but I still get this message. And in task explorer I see how "Microsoft.VSCode.CPP.IntelliSense.Msvc.exe" allocate memory and then it up to 4G is closed.
Reproducing steps is not clear. But rollback to 0.26.1 helps until restart the VCCode (I can mistake, need more investigation).
When I got clear steps, I will inform you. I'll try to get the "Microsoft.VSCode.CPP.IntelliSense.Msvc.exe" dump during crush.
My configuration is still the same:
{
"compileCommands": ".../compile_commands.json",
"compilerPath": ".../14.16.27023/bin/Hostx64/x64/cl.exe",
"intelliSenseMode": "msvc-x64",
"cppStandard": "c++17",
"cStandard": "c11",
"defines": [.....],
"includePath": [....].
"browse": {
"limitSymbolsToIncludedHeaders": true
}
}
Originally posted by @Gargony in #3270 (comment)
The text was updated successfully, but these errors were encountered: