Skip to content

[PoC] Make use of vscode.lm tools contributed by arbitrary VSCode extensions #684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

complyue
Copy link

Context

vscode.lm officially has agentic tool interface defined, "vscode.copilot-chat" provides most of GitHub Copilot tools through that interface, and there are many standalone tool-providing VSCode extensions out there: @tag:language-model-tools including my own: https://marketplace.visualstudio.com/items?itemName=ComplYue.roo-nb

This PR proves the concept for Kilo (and a sister PR for Roo) to call those tools right there.

p.s. Roo team is less inclined to discuss this idea in near future, wonder about Kilo's position here.

Implementation

Pls see https://github.com/complyue/kilocode/blob/vsclmt/src/services/vsclm/design.md

Screenshots

image

More tools can be added:

image

How to Test

Tell Kilo to use some of those tools selected.

Get in Touch

Discord: complyue

Copy link

changeset-bot bot commented Jun 11, 2025

⚠️ No Changeset found

Latest commit: 2377b26

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this delete should probably be reverted

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

definitely if Kilo would accept this PR.

it's deleted as tests have not been updated to pass, according to function/class signature changes for the functionalities. serve as a workaround to make it pushable for initial discussion.

the tests are less concert, compared to tool approval ui/logic, which would need major workouts beyond done so far.

anyway, let's see it working and discuss/decide further directions for now.

@drakonen drakonen requested a review from Copilot June 11, 2025 12:27
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces support for invoking VS Code LM tools via a new agentic interface, along with a persistent tool selection TreeView, and integrates these capabilities with Kilo's core systems.

  • Added new interfaces and configuration for VS Code LM tools
  • Implemented VSCLMToolsService to manage and expose available tools via a TreeView
  • Updated provider, prompt generation, and tool invocation logic to leverage the VSCLM tool system
  • Removed the pre-push hook file

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.

Show a summary per file
File Description
src/shared/tools.ts Added new interfaces and configuration for VS Code LM tools
src/services/vsclm/design.md Provided design details for the new tool selection feature
src/services/vsclm/VSCLMToolsService.ts Implemented the service to manage and display VSCLM tool data
src/package.json Registered new tree view and commands for tool selection
src/core/webview/generateSystemPrompt.ts Integrated VSCLM tool service into system prompt generation
src/core/webview/ClineProvider.ts Added VSCLMToolsService instance and accessor in the provider
src/core/tools/vsclmt.ts Created tool invocation routine for VS Code LM tools
src/core/task/Task.ts Included VSCLM service when generating system prompts
src/core/prompts/tools/vsclmt.ts Provided documentation for the new VSCLMT tool
src/core/prompts/tools/index.ts Registered the VSCLMT tool description mapping
src/core/prompts/system.ts Updated system prompt generation to include VSCLMT info
src/core/prompts/sections/vsclmt.ts Added a section to display available VS Code LM tools in prompts
src/core/prompts/sections/index.ts Exported the VSCLMT prompts section
src/core/assistant-message/presentAssistantMessage.ts Integrated the VSCLMT tool in the assistant message presentation
packages/types/src/tool.ts Updated tool types to include VSCLMT commands
.husky/pre-push Removed the pre-push hook that previously enforced pre-deployment checks
Comments suppressed due to low confidence (2)

.husky/pre-push:1

  • The removal of the pre-push hook may bypass important pre-deployment validations like type checking and changeset verification. Confirm that this removal is intentional or consider restoring the hook to maintain development safeguards.
diff --git a/.husky/pre-push b/.husky/pre-push

src/services/vsclm/VSCLMToolsService.ts:263

  • The prepareToolInvocation method always returns undefined, which prevents the handling of tool confirmation messages. Consider implementing the intended functionality or updating the documentation to remove potential confusion.
return undefined

Copy link
Collaborator

@kevinvandijk kevinvandijk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a POC this is definitely very cool, thank you! I think we should add this for sure but we need to tweak some more on this PR to make it compatible with how we want the UI for it to be, for example adding it as a separate pane in the activity bar would not be my preference. I'd rather have it be a settings pane or something like that. One reason being is that it now breaks the 'open in editor' mode but and having another pane below the default chat view is not ideal.

For the rest I would definitely like to land this! //cc @RSO

@complyue
Copy link
Author

@kevinvandijk @RSO Glad you would consider to land this!

If Kilo would like to merge this, I'm happy to further work it out wrt ideal UI/UX aspects.

Do you have guidance or best practice for the mods, as to ease pulling new features from Roo repo?

I had some pain experience to maintain XXL sized (due to UI mods involved) PR against Roo, that being ever (sometimes hardly) rebased with Roo's fast dev pace, this PoC mod by far is designed to be minimal wrt frequent rebasing in mind, so not taking UI/UX as a priority till now.

@complyue
Copy link
Author

@chrarnoldus Pls chime in and comment some?

@complyue complyue force-pushed the vsclmt branch 2 times, most recently from 1d2e508 to 2377b26 Compare June 17, 2025 14:29
@chrarnoldus
Copy link
Contributor

chrarnoldus commented Jun 17, 2025

Thanks for your patience @complyue. As @kevinvandijk mentioned, we are definitely interested in taking this. My feedback so far is the following:

  • Since it's a bigger feature, we probably want it under an experimental flag for now.
  • I think the tool selection window should be moved to settings (like mcp etc.), it does not need to be on the screen all the time.
    • Maybe auto-approve should be configurable per tool?
    • Currently I cannot change approval mode, I get an error "Unable to write to workspace settings".
  • The LLM seems to be reluctant to use the tools, unless you explicitly tell it to use the "vscode lm tool". Is that also your experience?

Are you interested in continuing to work on this? We can also consider taking it from here otherwise.

@kevinvandijk @hassoncs thoughts?

@adamhill
Copy link
Contributor

@complyue I just used your PR to tell GitViz to visualize commits.... kinda cool.

One anomaly though, I installed the GitHub Remote Server, the tools show up in Copilots' Tool dropdown, but not in your list. Any ideas why?

@adamhill
Copy link
Contributor

adamhill commented Jun 18, 2025

@chrarnoldus @kevinvandijk @hassoncs

In Roo they had to add codebase_search to several parts of the prompting system. This is going to be an ongoing challenge in the future. MS is going the way of using the lm.tools aggregation (namespace?) as one of their expansion point for tools in general. Esp, coming from MCP's.

There might be a need to make the whole tool system, discovery based and configurable (to use a tool or not) at many levels per project / monorepo folder / or per mode basis.

Copilot already has an inclusion method with toolsets and use the # chat marker to indicate their use in a prompt - https://code.visualstudio.com/updates/v1_101#_chat-tool-sets

They use //mcp.servername.promptname to surface MCP prompts, complete with varable insertion by input prompting. And Resources surfaced from an MCP (and they kinda look dynamic-y from the video) can just be dragged and dropped into the Chat.

MS has significantly raised the bar for they whole MCP client consumption ecosystem with the latest Copilot / VS Code release

Sounds like a fun project to start on 👨‍🚀 🚀

@complyue
Copy link
Author

@chrarnoldus

  • I think the tool selection window should be moved to settings (like mcp etc.), it does not need to be on the screen all the time.

I very much agree! the treeview solution in this PoC is solely for easier rebasing by myself to keep up with Kilo/Roo in building .vsix for my team use.

  • Maybe auto-approve should be configurable per tool?

I think it's possible, just set chat.tools.autoApprove to be a dict mapping tool names to booleans, rather than a simple boolean should do the same effect. yet this is not aligned with Kilo/Roo's approval mechanism that well.

  • Currently I cannot change approval mode, I get an error "Unable to write to workspace settings".

it'll write into .vscode/settings.json or xxx.code-workspace , might bcoz of file permissions?

  • The LLM seems to be reluctant to use the tools, unless you explicitly tell it to use the "vscode lm tool". Is that also your experience?

yes, pretty the same.
maybe core prompts should be tweaked to give lm tools the same awareness by agents, as other tools.

Are you interested in continuing to work on this? We can also consider taking it from here otherwise.

I tried to tackle the webview ui part, but it touches too many places in the codebase, e.g. extension messaging, those I don't think I can maintain well enough.

I would appreciate very much if you take it from here, you (+ @kevinvandijk @hassoncs) are definitely more capable than me to get this functionality into maintainable code. that said I'll be very glad to be in the loop and see what I can contribute.

@complyue
Copy link
Author

One anomaly though, I installed the GitHub Remote Server, the tools show up in Copilots' Tool dropdown, but not in your list. Any ideas why?

@adamhill I'm not sure which ext you are referring to? https://marketplace.visualstudio.com/search?term=github%20remote%20server&target=VSCode&category=All%20categories&sortBy=Relevance didn't show an exact match.

basically its package.json needs to contain sth under contributes.languageModelTools, as to be parsed at: https://github.com/complyue/kilocode/blob/2377b26cbce6abce3a926e08697d0932cade3612/src/services/vsclm/VSCLMToolsService.ts#L120

@adamhill
Copy link
Contributor

@complyue I am referring to The official GitHub Remote server - https://github.blog/changelog/2025-06-12-remote-github-mcp-server-is-now-available-in-public-preview/ the tools show up in Agent mode along with several other lm.tools

@complyue
Copy link
Author

@adamhill so they are probably MCP tools, this PoC is not discovering them by far, currently limited to LM tools from vscode extensions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants