Language Server Index Format

[dbaeumer]

The purpose of the Language Server Index Format (LSIF) is it to define a standard format for language servers or other programming tools to dump their knowledge about a workspace. This dump can later be used to answer language server LSP requests for the same workspace without running the language server itself. Since much of the information would be invalidated by a change to the workspace, the dumped information typically excludes requests used when mutating a document. So, for example, the result of a code complete request is typically not part of such a dump.

A first draft of a specification is available here

[rcjsuen]

@dbaeumer Not sure if I'm misinterpreting something here, but correct to say that this is something for LSP clients to implement and not servers to implement, yes?

[LaurentTreguier]

I understood it as something dumped by servers, and then used by clients afterwards, so both would have their part to implement

[rcjsuen]

Hm...good point. I had interpreted it as more of a caching system for the LSP client. However, I guess the first sentence should've made it clear to me... :(

The purpose of the Language Server Index Format (LSIF) is it to define a standard format for language servers or other programming tools to dump their knowledge about a workspace.

Well then...!

[dbaeumer]

@rcjsuen no the idea is that this is dumped either by the server or a separate tool. As mentioned in the spec I have already written a tools for TypeScript and an generic extension that serves the dump via LSP to any kind of LSP client. I will make these open source the next couple of days.

[tsmaeder]

So if I understand this correctly, this is supposed to be a library for helping language server implementers solve the problem of maintaining an index? If so, why does it need a specification as opposed to simply a documentation of the code? What are the reuse cases?

[dbaeumer]

Looks like I was pretty bad in explaining it: the goal is that we can produce an index to answer LSP requests for read only workspaces without firing up a language server specific to the programming language. There will be one generic language server that can serve the index. Furthermore the index will allow to relate symbols across repositories. See the demo here where Jonathan navigates from the use of observable to the actual definition in source.

[svenefftinge]

Sorry, still not sure I got it. Is it some sort of cache middleware for language servers?
If so why does it need to be part of the protocol?

[mickaelistria]

If so why does it need to be part of the protocol?

I am glad the LSP protocol also includes some technical proposals, middleware or intermediary formats to allow combination of different language servers.

So this index format will come with an implementation of a language server able to process multiple indexes to return results? How is this "composite index-based language server" expected to know which LS to retrive indexes from or how to get indexes?
And the LSP will be extended so that existing LS could provide the ability to dump and index and then shutdown assuming the index wouldn't have to change?

[svenefftinge]

to allow combination of different language servers

From where did you draw this would be supported? It would indeed be useful if language servers could access a common cross-languages index.

I am glad the LSP protocol also includes some technical proposals, middleware or intermediary formats

I was not implying it should not, but wanted to understand why and how it depends on the LSP (technically).

[dbaeumer]

Some clarifications: the LSIF will not be part of the protocol itself since it is not a protocol. What might be part of the protocol are requests to ask a server to dump it's state.

Why did we decide to put it here: the LSIF is based on LSP data types. The questions that are answerable by a dump are typical LSP requests useful on a read only workspace (for example goto definition, find all references).

Yes, we have developed a generic language server that can read in many indices and serve LSP requests on them. So it can serve a C# index in parallel to a TS index. I will make the TS index generator and the generic language server with an VS Code extension public soon. Will add a message here when available.

[dbaeumer]

Here we go:

• https://github.com/Microsoft/lsif-typescript for the TS specific index dumper
• https://github.com/Microsoft/vscode-lsif-extension for the extension. Please note that the extension is currently relying on proposed API and can therefore not be installed into VS Code directly

[tsmaeder]

I'm still a bit hazy on the motivation here: are you trying to solve a latency problem? Also, what's the use case for "read-only workpaces"?

[dbaeumer]

It is more about repositories and published version. In my projects I usually have dependencies to many other npm packages which I depend on on a certain version. To be able to navigate and browser them there is no need to spin a whole language server (no need for code complete, signature help, ...). If there would be an index it would be relatively cheap to serve these and to support navigating to them even without cloning the repository locally.

[dbaeumer]

Shown here to navigate from one source base to mobx sources on Github.