Generate JSON Schema and Markdown doc for .sourcekit-lsp/config.json

Providing a JSON schema for the configuration file should improve the
developer experience thanks to better auto-completion and diagnostics
support provided by some editors.

Additionally, we have been manually maintaining the configuration file
format documentation in `Documentation/Configuration File.md`, but it's
easy for the documentation to get out of sync with the actual schema.

This change introduces a new tool, `ConfigSchemaGen`, that generates a
JSON schema and a Markdown document for the configuration file based on
the Swift type definitions in the `SKOptions` module by using
swift-syntax.

Author: kateinoigakukun

Documentation/Configuration File.md

@@ -1,3 +1,5 @@
+<!-- DO NOT EDIT THIS FILE. This file is generated by ConfigSchemaGen/OptionDocument.swift. -->
+
 # Configuration File
 
 `.sourcekit-lsp/config.json` configuration files can be used to modify the behavior of SourceKit-LSP in various ways. The following locations are checked. Settings in later configuration files override settings in earlier configuration files
@@ -12,44 +14,64 @@ The structure of the file is currently not guaranteed to be stable. Options may
 ## Structure
 
 `config.json` is a JSON file with the following structure. All keys are optional and unknown keys are ignored.
-
-- `swiftPM`: Dictionary with the following keys, defining options for SwiftPM workspaces
-  - `configuration: "debug"|"release"`: The configuration to build the project for during background indexing and the configuration whose build folder should be used for Swift modules if background indexing is disabled. Equivalent to SwiftPM's `--configuration` option.
-  - `scratchPath: string`: Build artifacts directory path. If nil, the build system may choose a default value. This path can be specified as a relative path, which will be interpreted relative to the project root. Equivalent to SwiftPM's `--scratch-path` option.
+- `swiftPM`: Options for SwiftPM workspaces
+  - `configuration: "debug"|"release"`: The configuration to build the project for during background indexing
+    and the configuration whose build folder should be used for Swift
+    modules if background indexing is disabled.
+    Equivalent to SwiftPM's `--configuration` option.
+  - `scratchPath: string`: Build artifacts directory path. If nil, the build system may choose a default value.
+    This path can be specified as a relative path, which will be interpreted relative to the project root.
+    Equivalent to SwiftPM's `--scratch-path` option.
   - `swiftSDKsDirectory: string`: Equivalent to SwiftPM's `--swift-sdks-path` option
   - `swiftSDK: string`: Equivalent to SwiftPM's `--swift-sdk` option
   - `triple: string`: Equivalent to SwiftPM's `--triple` option
   - `cCompilerFlags: string[]`: Extra arguments passed to the compiler for C files. Equivalent to SwiftPM's `-Xcc` option.
   - `cxxCompilerFlags: string[]`: Extra arguments passed to the compiler for C++ files. Equivalent to SwiftPM's `-Xcxx` option.
   - `swiftCompilerFlags: string[]`: Extra arguments passed to the compiler for Swift files. Equivalent to SwiftPM's `-Xswiftc` option.
   - `linkerFlags: string[]`: Extra arguments passed to the linker. Equivalent to SwiftPM's `-Xlinker` option.
-  - `disableSandbox: bool`: Disables running subprocesses from SwiftPM in a sandbox. Useful when running `sourcekit-lsp` in a sandbox because nested sandboxes are not supported.
+  - `disableSandbox: boolean`: Disables running subprocesses from SwiftPM in a sandbox. Equivalent to SwiftPM's `--disable-sandbox` option
+    Useful when running `sourcekit-lsp` in a sandbox because nested sandboxes are not supported.
 - `compilationDatabase`: Dictionary with the following keys, defining options for workspaces with a compilation database
   - `searchPaths: string[]`: Additional paths to search for a compilation database, relative to a workspace root.
 - `fallbackBuildSystem`: Dictionary with the following keys, defining options for files that aren't managed by any build system
   - `cCompilerFlags: string[]`: Extra arguments passed to the compiler for C files
   - `cxxCompilerFlags: string[]`: Extra arguments passed to the compiler for C++ files
   - `swiftCompilerFlags: string[]`: Extra arguments passed to the compiler for Swift files
   - `sdk: string`: The SDK to use for fallback arguments. Default is to infer the SDK using `xcrun`.
-- `buildSettingsTimeout: int`: Number of milliseconds to wait for build settings from the build system before using fallback build settings.
+- `buildSettingsTimeout: integer`: Number of milliseconds to wait for build settings from the build system before using fallback build settings.
 - `clangdOptions: string[]`: Extra command line arguments passed to `clangd` when launching it
-- `index`: Dictionary with the following keys, defining options related to indexing
+- `index`: Options related to indexing
   - `indexStorePath: string`: Directory in which a separate compilation stores the index store. By default, inferred from the build system.
   - `indexDatabasePath: string`: Directory in which the indexstore-db should be stored. By default, inferred from the build system.
   - `indexPrefixMap: [string: string]`: Path remappings for remapping index data for local use.
-  - `maxCoresPercentageToUseForBackgroundIndexing: double`: A hint indicating how many cores background indexing should use at most (value between 0 and 1). Background indexing is not required to honor this setting
-  - `updateIndexStoreTimeout: int`: Number of seconds to wait for an update index store task to finish before killing it.
-- `logging`: Dictionary with the following keys, changing SourceKit-LSP’s logging behavior on non-Apple platforms. On Apple platforms, logging is done through the [system log](Diagnose%20Bundle.md#Enable%20Extended%20Logging). These options can only be set globally and not per workspace.
-  - `logLevel: "debug"|"info"|"default"|"error"|"fault"`: The level from which one onwards log messages should be written.
-  - `privacyLevel: "public"|"private"|"sensitive"`: Whether potentially sensitive information should be redacted. Default is `public`, which redacts potentially sensitive information.
-  - `inputMirrorDirectory: string`: Write all input received by SourceKit-LSP on stdin to a file in this directory. Useful to record and replay an entire SourceKit-LSP session.
-- `defaultWorkspaceType: "buildserver"|"compdb"|"swiftpm"`: Overrides workspace type selection logic.
+  - `maxCoresPercentageToUseForBackgroundIndexing: number`: A hint indicating how many cores background indexing should use at most (value between 0 and 1). Background indexing is not required to honor this setting
+  - `updateIndexStoreTimeout: integer`: Number of seconds to wait for an update index store task to finish before killing it.
+- `logging`: Options related to logging, changing SourceKit-LSP’s logging behavior on non-Apple platforms.
+  On Apple platforms, logging is done through the [system log](Diagnose%20Bundle.md#Enable%20Extended%20Logging).
+  These options can only be set globally and not per workspace.
+  - `level: "debug"|"info"|"default"|"error"|"fault"`: The level from which one onwards log messages should be written.
+  - `privacyLevel: "sensitive"|"private"|"public"`: Whether potentially sensitive information should be redacted.
+    Default is `public`, which redacts potentially sensitive information.
+  - `inputMirrorDirectory: string`: Write all input received by SourceKit-LSP on stdin to a file in this directory.
+    Useful to record and replay an entire SourceKit-LSP session.
+- `defaultWorkspaceType: "buildServer"|"compilationDatabase"|"swiftPM"`: Default workspace type (buildserver|compdb|swiftpm). Overrides workspace type selection logic.
 - `generatedFilesPath: string`: Directory in which generated interfaces and macro expansions should be stored.
-- `backgroundIndexing: bool`: Explicitly enable or disable background indexing.
-- `backgroundPreparationMode: "build"|"noLazy"|"enabled"`: Determines how background indexing should prepare a target. Possible values are:
+- `backgroundIndexing: boolean`: Whether background indexing is enabled.
+- `backgroundPreparationMode: string`: Determines how background indexing should prepare a target. Possible values are:
   - `build`: Build a target to prepare it
   - `noLazy`: Prepare a target without generating object files but do not do lazy type checking and function body skipping
   - `enabled`: Prepare a target without generating object files and the like
-- `cancelTextDocumentRequestsOnEditAndClose: bool`: Whether sending a `textDocument/didChange` or `textDocument/didClose` notification for a document should cancel all pending requests for that document.
-- `experimentalFeatures: string[]`: Experimental features to enable. Available features: on-type-formatting
-- `swiftPublishDiagnosticsDebounceDuration: double`: The time that `SwiftLanguageService` should wait after an edit before starting to compute diagnostics and sending a `PublishDiagnosticsNotification`.
+- `cancelTextDocumentRequestsOnEditAndClose: boolean`: Whether sending a `textDocument/didChange` or `textDocument/didClose` notification for a document should cancel
+  all pending requests for that document.
+- `experimentalFeatures: ("on-type-formatting")[]`: Experimental features that are enabled.
+- `swiftPublishDiagnosticsDebounceDuration: number`: The time that `SwiftLanguageService` should wait after an edit before starting to compute diagnostics and
+  sending a `PublishDiagnosticsNotification`.
+- `workDoneProgressDebounceDuration: number`: When a task is started that should be displayed to the client as a work done progress, how many milliseconds to
+  wait before actually starting the work done progress. This prevents flickering of the work done progress in the
+  client for short-lived index tasks which end within this duration.
+- `sourcekitdRequestTimeout: number`: The maximum duration that a sourcekitd request should be allowed to execute before being declared as timed out.
+  In general, editors should cancel requests that they are no longer interested in, but in case editors don't cancel
+  requests, this ensures that a long-running non-cancelled request is not blocking sourcekitd and thus most semantic
+  functionality.
+  In particular, VS Code does not cancel the semantic tokens request, which can cause a long-running AST build that
+  blocks sourcekitd.

Sources/SKOptions/SourceKitLSPOptions.swift

@@ -31,13 +31,16 @@ import struct TSCBasic.AbsolutePath
 /// See `ConfigurationFile.md` for a description of the configuration file's behavior.
 public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
   public struct SwiftPMOptions: Sendable, Codable, Equatable {
-    /// Build configuration (debug|release).
+    /// The configuration to build the project for during background indexing
+    /// and the configuration whose build folder should be used for Swift
+    /// modules if background indexing is disabled.
     ///
     /// Equivalent to SwiftPM's `--configuration` option.
     public var configuration: BuildConfiguration?
 
     /// Build artifacts directory path. If nil, the build system may choose a default value.
     ///
+    /// This path can be specified as a relative path, which will be interpreted relative to the project root.
     /// Equivalent to SwiftPM's `--scratch-path` option.
     public var scratchPath: String?
 
@@ -50,19 +53,20 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
     /// Equivalent to SwiftPM's `--triple` option
     public var triple: String?
 
-    /// Equivalent to SwiftPM's `-Xcc` option
+    /// Extra arguments passed to the compiler for C files. Equivalent to SwiftPM's `-Xcc` option.
     public var cCompilerFlags: [String]?
 
-    /// Equivalent to SwiftPM's `-Xcxx` option
+    /// Extra arguments passed to the compiler for C++ files. Equivalent to SwiftPM's `-Xcxx` option.
     public var cxxCompilerFlags: [String]?
 
-    /// Equivalent to SwiftPM's `-Xswiftc` option
+    /// Extra arguments passed to the compiler for Swift files. Equivalent to SwiftPM's `-Xswiftc` option.
     public var swiftCompilerFlags: [String]?
 
-    /// Equivalent to SwiftPM's `-Xlinker` option
+    /// Extra arguments passed to the linker. Equivalent to SwiftPM's `-Xlinker` option.
     public var linkerFlags: [String]?
 
-    /// Equivalent to SwiftPM's `--disable-sandbox` option
+    /// Disables running subprocesses from SwiftPM in a sandbox. Equivalent to SwiftPM's `--disable-sandbox` option
+    /// Useful when running `sourcekit-lsp` in a sandbox because nested sandboxes are not supported.
     public var disableSandbox: Bool?
 
     public init(
@@ -122,9 +126,13 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
   }
 
   public struct FallbackBuildSystemOptions: Sendable, Codable, Equatable {
+    /// Extra arguments passed to the compiler for C files
     public var cCompilerFlags: [String]?
+    /// Extra arguments passed to the compiler for C++ files
     public var cxxCompilerFlags: [String]?
+    /// Extra arguments passed to the compiler for Swift files
     public var swiftCompilerFlags: [String]?
+    /// The SDK to use for fallback arguments. Default is to infer the SDK using `xcrun`.
     public var sdk: String?
 
     public init(
@@ -153,10 +161,15 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
   }
 
   public struct IndexOptions: Sendable, Codable, Equatable {
+    /// Directory in which a separate compilation stores the index store. By default, inferred from the build system.
     public var indexStorePath: String?
+    /// Directory in which the indexstore-db should be stored. By default, inferred from the build system.
     public var indexDatabasePath: String?
+    /// Path remappings for remapping index data for local use.
     public var indexPrefixMap: [String: String]?
+    /// A hint indicating how many cores background indexing should use at most (value between 0 and 1). Background indexing is not required to honor this setting
     public var maxCoresPercentageToUseForBackgroundIndexing: Double?
+    /// Number of seconds to wait for an update index store task to finish before killing it.
     public var updateIndexStoreTimeout: Int?
 
     public var maxCoresPercentageToUseForBackgroundIndexingOrDefault: Double {
@@ -202,6 +215,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
     public var level: String?
 
     /// Whether potentially sensitive information should be redacted.
+    /// Default is `public`, which redacts potentially sensitive information.
     public var privacyLevel: String?
 
     /// Write all input received by SourceKit-LSP on stdin to a file in this directory.
@@ -241,18 +255,21 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
     case enabled
   }
 
+  /// Options for SwiftPM workspaces
   private var swiftPM: SwiftPMOptions?
   public var swiftPMOrDefault: SwiftPMOptions {
     get { swiftPM ?? .init() }
     set { swiftPM = newValue }
   }
 
+  /// Dictionary with the following keys, defining options for workspaces with a compilation database
   private var compilationDatabase: CompilationDatabaseOptions?
   public var compilationDatabaseOrDefault: CompilationDatabaseOptions {
     get { compilationDatabase ?? .init() }
     set { compilationDatabase = newValue }
   }
 
+  /// Dictionary with the following keys, defining options for files that aren't managed by any build system
   private var fallbackBuildSystem: FallbackBuildSystemOptions?
   public var fallbackBuildSystemOrDefault: FallbackBuildSystemOptions {
     get { fallbackBuildSystem ?? .init() }
@@ -266,14 +283,20 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
     get { .milliseconds(buildSettingsTimeout ?? 500) }
   }
 
+  /// Extra command line arguments passed to `clangd` when launching it
   public var clangdOptions: [String]?
 
+  /// Options related to indexing
   private var index: IndexOptions?
   public var indexOrDefault: IndexOptions {
     get { index ?? .init() }
     set { index = newValue }
   }
 
+  /// Options related to logging, changing SourceKit-LSP’s logging behavior on non-Apple platforms.
+  ///
+  /// On Apple platforms, logging is done through the [system log](Diagnose%20Bundle.md#Enable%20Extended%20Logging).
+  /// These options can only be set globally and not per workspace.
   private var logging: LoggingOptions?
   public var loggingOrDefault: LoggingOptions {
     get { logging ?? .init() }
@@ -282,6 +305,7 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
 
   /// Default workspace type (buildserver|compdb|swiftpm). Overrides workspace type selection logic.
   public var defaultWorkspaceType: WorkspaceType?
+  /// Directory in which generated interfaces and macro expansions should be stored.
   public var generatedFilesPath: String?
 
   /// Whether background indexing is enabled.
@@ -291,6 +315,10 @@ public struct SourceKitLSPOptions: Sendable, Codable, Equatable {
     return backgroundIndexing ?? true
   }
 
+  /// Determines how background indexing should prepare a target. Possible values are:
+  ///   - `build`: Build a target to prepare it
+  ///   - `noLazy`: Prepare a target without generating object files but do not do lazy type checking and function body skipping
+  ///   - `enabled`: Prepare a target without generating object files and the like
   public var backgroundPreparationMode: String?
 
   public var backgroundPreparationModeOrDefault: BackgroundPreparationMode {

Utilities/ConfigSchemaGen/.gitignore

@@ -0,0 +1,8 @@
+.DS_Store
+/.build
+/Packages
+xcuserdata/
+DerivedData/
+.swiftpm/configuration/registries.json
+.swiftpm/xcode/package.xcworkspace/contents.xcworkspacedata
+.netrc

Utilities/ConfigSchemaGen/Package.swift

@@ -0,0 +1,20 @@
+// swift-tools-version: 6.0
+
+import PackageDescription
+
+let package = Package(
+  name: "ConfigSchemaGen",
+  platforms: [.macOS(.v10_15)],
+  dependencies: [
+    .package(url: "https://github.com/swiftlang/swift-syntax.git", from: "600.0.1")
+  ],
+  targets: [
+    .executableTarget(
+      name: "ConfigSchemaGen",
+      dependencies: [
+        .product(name: "SwiftSyntax", package: "swift-syntax"),
+        .product(name: "SwiftParser", package: "swift-syntax"),
+      ]
+    )
+  ]
+)

Utilities/ConfigSchemaGen/README.md

@@ -0,0 +1,14 @@
+# Configuration Schema Generation for SourceKit-LSP
+
+This directory contains a tool to generate a JSON schema and Markdown
+documentation for the SourceKit-LSP configuration file format
+(`.sourcekit-lsp/config.json`) from the Swift type definitions in `SKOptions`
+Swift module.
+
+To regenerate the schema and documentation, run the following command from the
+root of the repository:
+
+```sh
+swift run --package-path Utilities/ConfigSchemaGen
+```
+