Large completion lists (e.g. typing "Se") break language services in VS Code on Windows

[tristanlabelle]

After some completion requests, language services stop responding (can't ctrl+click, hover says "Loading..." forever, no more completion, etc). This makes SourceKit-LSP hardly usable and seems like a recent-ish regression (~3 weeks?)

Repro

1. Open a file and type

let foo = 0
S

2. Hover over foo, its description pops up
3. After S, type 7
4. Hover over foo, its description still pops up
5. Erase 7 and type e
6. Hover over foo, see Loading... forever

Expected

Language services continue working.

Environment

Swift:

Swift version 5.9-dev (LLVM ccba7f42fded680, Swift f5fbee2f26addeb)
Target: x86_64-unknown-windows-msvc

VSCode:

Version: 1.78.2 (user setup)
Commit: b3e4e68a0bc097f0ae7907b217c1119af9e03435
Date: 2023-05-10T14:39:26.248Z (4 wks ago)
Electron: 22.5.2
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: 10.8.168.25-electron.0
OS: Windows_NT x64 10.0.22621
Sandboxed: No

Swift extension:

Name: Swift
Id: sswg.swift-lang
Description: Swift Language Support for Visual Studio Code.
Version: 1.3.0
Publisher: Swift Server Work Group
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=sswg.swift-lang

Investigation

I locally rebuilt SourceKit-LSP with only support for textDocument/completion and textDocument/hover and can repro this (except that I don't get squiggles, expectedly). However I cannot repro it after disabling textDocument/completion.

Curiously, SourceKit-LSP keeps logging valid request/response pairs, but it seems like VSCode never processes them.

The only difference in LSP logging between the S7 and Se cases are:

• Se results in a massive completion response (173 items), whereas S7 has an empty response
• VS Code sends $/cancelRequests after the completion of the hover request in the S7 case:

sourcekit-lsp.exe SourceKitServer: Notification<$/cancelRequest>(
  clientID: ObjectIdentifier,
  params: CancelRequestNotification(id: 2)
)
sourcekit-lsp.exe SourceKitServer: Notification<$/cancelRequest>(
  clientID: ObjectIdentifier,
  params: CancelRequestNotification(id: 1)
)

When artificially trimming down the completion response to 50 items in SourceKit-LSP, the issue disappears, even if adding a sleep, so this has to do with the payload size of large completion lists. I dumped the JSON and it is well-formatted to this is not the issue.

Testing different completions and monitoring the size of the resulting json response, it seems like something funny happens around 0xFFFF:

// Not broken: 54362 (Sa)
// Not broken: 63208 (Sr)
// Broken: 69602 (St)
// Broken: 79583 (Se)

[tristanlabelle]

(I'm investigating this issue, filing here for visibility in case someone has a hunch)

[ahoppen]

Tracked in Apple’s issue tracker as rdar://110399252

[tristanlabelle]

It might have to do with $/cancelRequest interplay with a large textDocument/completion requests?

sourcekit-lsp.exe SourceKitServer: Request<textDocument/completion>(
  id: 2,
  clientID: ObjectIdentifier(0x000001f6c6888970),
  params: CompletionRequest(textDocument: LanguageServerProtocol.TextDocumentIdentifier(uri: LanguageServerProtocol.DocumentURI(storage: file:///c%3A/Users/tristan/Dev/swift-playground/unresponsive.swift)), position: Position(line: 2, utf16index: 14), context: Optional(LanguageServerProtocol.CompletionContext(triggerKind: LanguageServerProtocol.CompletionTriggerKind(rawValue: 3), triggerCharacter: nil)), sourcekitlspOptions: nil)
)
...
sourcekit-lsp.exe SourceKitServer: Response<textDocument/completion(2)>(
<huge response>
)
...
sourcekit-lsp.exe SourceKitServer: Notification<$/cancelRequest>(
  clientID: ObjectIdentifier(0x000001f6c6888970),
  params: CancelRequestNotification(id: 2)
)
...
sourcekit-lsp.exe SourceKitServer: Request<textDocument/hover>(
  id: 3,
  clientID: ObjectIdentifier(0x000001f6c6888970),
  params: HoverRequest(textDocument: LanguageServerProtocol.TextDocumentIdentifier(uri: LanguageServerProtocol.DocumentURI(storage: file:///c%3A/Users/tristan/Dev/swift-playground/unresponsive.swift)), position: Position(line: 2, utf16index: 19))
)
...
sourcekit-lsp.exe SourceKitServer: Response<textDocument/hover(3)>(
  success(Optional(LanguageServerProtocol.HoverResponse(contents: LanguageServerProtocol.HoverResponseContents.markupContent(LanguageServerProtocol.MarkupContent(kind: LanguageServerProtocol.MarkupKind(rawValue: "markdown"), value: "foo\n```swift\nstatic let foo: Int\n```\n\n---\n")), _range: LanguageServerProtocol.CustomCodable<Swift.Optional<LanguageServerProtocol.PositionRange>>(wrappedValue: nil))))
)
sourcekit-lsp.exe SourceKitServer: Notification<$/cancelRequest>(
  clientID: ObjectIdentifier(0x000001f6c6888970),
  params: CancelRequestNotification(id: 3)
)

[ahoppen]

Could you check if sourcekitd is busy consuming CPU resources?

[tristanlabelle]

@ahoppen This is most likely a Windows-only bug in the Foundation.Pipe and DispatchIO implementation.

[ahoppen]

Oh, that’s… bad. Unfortunately I can’t help with that 😢

[adam-fowler]

Just so you know I cannot repro on macOS

[tristanlabelle]

@adam-fowler thanks, that helps confirm the diagnostic. I think fix won't be in this repo so if it's better to close this issue, I'm ok with that. Otherwise I'll close it once I fix the underlying problem.