Add a MCP server tool to retrieve the top package(s) for a use case

[kenzieschmoll]

Create an MCP server tool get_recommended_packages that can return a list of N package names, where N is arbitrarily 0-3 (see discussion items below).

Tool input:
The input spec is up for discussion (see discussion items below). IMO, option 2 is likely the proper approach if the LLM can help distill user input into categories the MCP tool can use as search queries.

Possible options:

1. A textual description of the package functionality. Some example inputs: "provides graphs for displaying various forms of data", "helps with writing and inspecting tests", "utilities for financial and economic math", etc.)
2. A list of categories created by the LLM. The LLM would take human input like the examples provided in option 1 and distill it down into categories or keywords that the MCP server tool can understand easier. Using the examples above, the example inputs would instead be ["graphs", "data visualization"], ["testing"], ["finance", "economics", "mathematical computation"].

Tool response:
Structured data including the recommended packages and their associated confidence score (how good of a match the package is for the query). This response, of course, should fit into the CallToolResult spec provided by the dart_mcp APIs.

{
  "foo": 0.91,
  "bar": 0.82,
  "baz": 0.70
}

Discussion items:

1. Should the input be a raw description provided by a user or a category / keyword list generated by an LLM?
2. How many packages should be returned by the tool?
   • We should limit this to some maximum (perhaps 3?) that will allow for alternative solutions if the top recommended package is incompatible with the current project, but that doesn't include low confidence matches.
   • We should prefer returning 0 packages over returning a package that isn't a moderate-to-high confidence match for the query.
   • Items should only be included in the list if they meet some minimum confidence threshold.

Tool implementation:

• The tool should take into account things like popularity, Flutter favorite status, pub score, download count, etc to return the top N packages for the query.
  • What other criteria should determine recommendation weight?
• The tool should also be aware of the supported platforms for the project (for example it should not return a web-only package for an app that only supports iOS and Android). This may be an input parameter for the tool or the tool should look this information up from the pubspec.yaml files available in the associated project.

Please provide feedback / edit this spec as needed: @gspencergoog @sigurdm @jonasfj @jakemac53 @natebosch @anderdobo

[jonasfj]

If we intended to leverage the search functionality we have on pub.dev, which I believe is exposed as a private API, I think we have an overall relevance score.

If giving the LLM too many packages is a concern, then perhaps we should be dumb and only tell it about top 10% most popular packages. Or even just flutter favorite packages.

Another option is to give the LLM a list of topics to pick from:
https://pub.dev/topics

Not all popular packages have topics yet, but many does.

[gspencergoog]

If we use topics, it would be good to include fetching the list of possible topics in the MCP API.

[natebosch]

only tell it about top 10% most popular packages. Or even just flutter favorite packages.

If we filter we should be very clear about when and why the filtering is happening. If a user finds 0 packages through the search MCP tool and they move to pub.dev with the same search and see results they will assume it's a bug.

[jakemac53]

In the tool response should we return more information about each package such as its API or the package examples?

This could aid the AI in determining which package is the best fit, as well as help it to successfully use the package.

[gspencergoog]

In the tool response should we return more information about each package such as its API or the package examples?

I think those things should be available, but we shouldn't return them by default, so the LLM gets only what it asked for (to keep the token usage to a minimum).

[jakemac53]

I think those things should be available, but we shouldn't return them by default, so the LLM gets only what it asked for (to keep the token usage to a minimum).

I think the question is whether or not the LLM is going to immediately ask for these things anyways. If it is, then we might as well return it as a part of the response. Especially if it wants to use that info to help choose between the options.

That will end up being cheaper overall (one or more fewer inferences)

[gspencergoog]

I think the question is whether or not the LLM is going to immediately ask for these things anyways. If it is, then we might as well return it as a part of the response. Especially if it wants to use that info to help choose between the options.

I don't think we can say whether the LLM will immediately request them: it may just be building a pubspec and not doing any coding with the package yet, so it just needs the package name and version.

You could just make it an optional argument to the tool call so that the LLM could decide if the extra information is useful or not and still be able to avoid unnecessary tokens and inferences.