High memory usage

Description

I have recently tried out the stable tool-chain (installed with swiftly on Manjaro Linux) with VSCode and its Swift plugin.
The memory usage of sourcekit-lsp easily freezes out the machine.

The machine has 16 CPU threads (8 core with 2 HW threads), but only 8GB RAM.
As I suspect the number of sourcekit-lsp threads scales up based on the number of CPU threads.

Is it possible to explicitly limit the sourcekit-lsp threads?
(Linux allows to turn off some cores, but that is not too convenient.)

Or would it be possible to also consider available RAM too when spawning sourcekit-lsp threads?
E.g. for 8GB of RAM only spawn 3 threads, even if the machine have 256 CPUs.

Synced to Apple’s issue tracker as rdar://130844901

@ebak

Hey @ebak,

A couple of questions:

How much memory does sourcekit-lsp use on your machine? I would be expecting something <500MB on start up, fairly independent of your CPU core count.
Does the memory usage of SourceKit-LSP grow over time or does it go up immediately after launching
Does this also reproduce with a Swift 6 toolchain. We have made significant changes to SourceKit-LSP between 5.10 and 6.0, so the issue might already be fixed.
Does the memory usage go up on every project that you open or only on specific ones? If so, would you be able to share the project?

Hi,

Now I don't have possibility to try it on the machine with 8 CPUs, maybe next week.

How much memory does sourcekit-lsp use on your machine? I would be expecting something <500MB on start up, fairly independent of your CPU core count.

I attach screenshots of htop output, as I see at startup it only consumes 2.8% (I don't know what htop means about this percentage.)

Does the memory usage of SourceKit-LSP grow over time or does it go up immediately after launching

Grows over time. As I see build and source browsing is required for it.

Does this also reproduce with a Swift 6 toolchain. We have made significant changes to SourceKit-LSP between 5.10 and 6.0, so the issue might already be fixed.

Not yet tried out, maybe I'll do when I have possibility to play it around with the system with 8 CPUs.

Does the memory usage go up on every project that you open or only on specific ones? If so, would you be able to share the project?

Recently I was working with this project (QtTest):
QtTest.zip

Just by editing the same code, the memory usage quickly goes up to 45% (3.6GB maybe):

Maybe unrelated but VSCode usually forgets about the dependent packages:

The dependencies (Qlift, Math) are sometimes shown up there, sometimes not.

Sad the 8GB RAM is hardly enough for IDE assisted development.
In my case the opened apps almost use up all free memory (Gnome, 2 terminals, 2 file managers, a light weight editor (gedit), an ebook reader for the Swift book, VSCode with Swift instrumentation).
Opening up a Firefox with 1-2 tabs would easily freeze out the system.

I have been able to reproduce the memory growth by working on the SourceKit-LSP codebase in VS Code on Ubuntu 22.04.

Thanks! With the 6.0 or the 5.10 toolchain? Maybe if this issue doesn't exist in 6.0, this issue could be closed and wait for 6.0 to get stable. (Currently I cannot test.)

I saw the memory issue with a 6.0 toolchain.

[DO NOT MERGE] Use default cache on macOS

i think that this issue is happening more after updating to Xcode 16.
it keeps growing until the whole device get freezed.

i downloaded Xcode 15, copied the sourceKit inside it, then placed it inside Xcode 16, and it appears to be working, and fixed my issue [until now]

Oh, that’s interesting that you’re also seeing excessive memory usage on macOS. I haven't seen that before.

Could you try reproducing the problem, run leaks --outputGraph=/tmp sourcekit-lsp and then share the .memgraph file that is being generated the command? I’m hoping that will show us where the memory allocations are from.

@ahoppen

@ahoppen i will try, when it happens i'm running against time to kill SourceKit-lsp before my device is freezed 🥶

i can also confirm that since i started using sourcekit-lsp from Xcode 15 the issue isn't appearing

I think it would be valuable to see the graph when SourceKit-LSP is at ~2GB. That’s already more than it should take and should contain the culprit who’s using the memory.

@ahoppen

@ahoppen
i just shared it with u over email, thanks for u

@iMostfa

Thanks for sharing the memgraph, @iMostfa 🙏🏽 With it, it was obvious that the failure cause is the same one as #1720.

Use an `AtomicInt32` to count `pendingUnitCount` instead of using `As…

I tracked down the memory leak to a bug in libdispatch. The issue should be fixed by swiftlang/swift-corelibs-libdispatch#850.

@ahoppen

This error prevents me from working on Xcode 16.2.

@ahoppen Is there a chance to fix this bug in Xcode 16.x or do we have to wait for Xcode 17?

I can confirm that the workaround provided by @iMostfa works (thanks a lot!).

More detailed instructions:

Disable System Integrity Protection: Apple Documentation
Get Xcode 15.4: Download Link (requires login to a developer account) and unzip archive.
Replace sourcekit-lsp binary:

sudo cp /<PATH_TO_XCODE_15.4>/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/sourcekit-lsp /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/

Please note that Xcode will only work with System Integrity Protection disabled.

@m-buczek

@m-buczek it's interesting that you needed SIP to be disabled, for me it worked without disabling it.
btw @ahoppen can you confirm to us if the changes to sourcekit-lsp were merged to Xcode[or swift which later merged to Xcode]

Thanks for also confirming that my PR above fixes the issue. The fix will be included in Swift 6.1. I can’t speak for Xcode’s release schedule.

Is anyone still having problems with this?

I am using Swift 6.1 installed via swiftly on Fedora 40 and sourcekit-lsp is heavily eating up my memory when working in Swift using VS Code or Zed.

The leaked memory seems to result mostly from operator new(unsigned long, std::align_val_t) in libstdc++.so.6.

@terraputix

@terraputix That seems surprising to me, that would imply we're leaking entire ASTContexts. Do you have a project you can share that reproduces the issue?

i can confirm that something is leaking again, on Mac OS, sourcekit-lsp took up 48GB of ram.
unfortunately, i wasn't able to get a mem dump because the whole system freezed

@iMostfa

The macOS and Linux leak are probably unrelated because the Linux leak seems to be coming from sourcekitd. macOS runs sourcekitd out-of-process, so the leak here must be coming from a leak inside SourceKit-LSP itself.

@iMostfa and @terraputix Could you file separate issues for the leaks you are seeing and include the following information?

Which version of SourceKit-LSP are you using?
Is this issue happening for one particular project / on one specific interaction or always? If it is one project, can you share it?
Do you have a rough rate at which it is leaking? Eg. does it take a minute to leak 1 GB, 5 minutes, one hour, etc.
@iMostfa If you could take a memory graph when sourcekit-lsp is using, say 10GB, on macOS as described in https://github.com/swiftlang/sourcekit-lsp/blob/main/Contributor%20Documentation/Debugging%20Memory%20Leaks.md#macos, that would be extremely helpful.

@ahoppen

@ahoppen, @hamishknight I created a follow up issue with the desired information in swiftlang/swift#82362.

I want to mention I've also been experiencing a lot of leaks for a long time. I was blaming it on Cursor and simply reopen Cursor to fix, but lately noticed It's actually not Cursor. I'll be trying to catch the leak and see if it matches any of the cases reported above.

Just a report: I haven't noticed something that I could call a "leak", recently, on macOS.
The memory usage is still somewhat high (~6-12GB in a M1 16GB machine - swap is definitely saving me) but it doesn't just go up indefinitely.

Navigation Menu

Repository navigation

High memory usage #1541

Description

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

High memory usage #1541

Description

Description

Activity

ahoppen commented on Jun 30, 2024

ahoppen commented on Jul 2, 2024

ebak commented on Jul 2, 2024

ebak commented on Jul 2, 2024

ahoppen commented on Jul 4, 2024

ebak commented on Jul 4, 2024

ahoppen commented on Jul 15, 2024

iMostfa commented on Sep 28, 2024

iMostfa commented on Sep 30, 2024

ahoppen commented on Oct 2, 2024

iMostfa commented on Oct 2, 2024

ahoppen commented on Oct 2, 2024

iMostfa commented on Oct 8, 2024

ahoppen commented on Oct 9, 2024

ahoppen commented on Oct 29, 2024

m-buczek commented on Jan 13, 2025

iMostfa commented on Jan 13, 2025

ahoppen commented on Jan 13, 2025

terraputix commented on May 22, 2025

hamishknight commented on May 28, 2025

iMostfa commented on Jun 7, 2025

ahoppen commented on Jun 10, 2025

terraputix commented on Jun 13, 2025

MahdiBM commented on Aug 29, 2025

MahdiBM commented on Sep 29, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions