Skip to content

A 'pub-dev-search' mcp tool #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open

Conversation

sigurdm
Copy link

@sigurdm sigurdm commented May 2, 2025

Adds a 'pub-dev-search' tool to the mcp tool list.

An attempt at #93

It strings together a number of (more or less secret) api calls from pub.dev to find relevant packages and information about them:

  • api/search/q= for finding a list of packages (limits to 10 packages).
  • api/packages/<package> for finding the description and latest version
  • api/packages/<package>/score for finding scores and tags
  • documentation/<package>/latest/index.json for listing top-level identifiers

Copy link

github-actions bot commented May 2, 2025

PR Health

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

@sigurdm
Copy link
Author

sigurdm commented May 2, 2025

Example transcript:

How can I help you today?

Can you find a dart package for modifying yaml? It should be a well supported package, and with liberal license.

Okay, here's my plan to find a suitable Dart package for modifying YAML:

  1. Search pub.dev: I will use the pub-dev-search tool with the search query "yaml" to find relevant packages.
  2. Analyze Results: I will examine the search results, focusing on packages that explicitly mention YAML modification or editing capabilities. I'll prioritize packages with high popularity (download counts), good maintenance indicators (like pub points), and liberal licenses (such as MIT, BSD, or Apache).
  3. Recommend Package: Based on the analysis, I will recommend the most suitable package that meets your criteria.
  4. Present Findings: I will share the details of the recommended package(s) with you.

Do you want me to proceed with this plan?

yes

Okay, executing the plan.

Step 1: Search pub.dev

I will now search pub.dev for packages related to "yaml".

I am going to run the pub-dev-search tool to perform this task.

Okay, proceeding to the next step.

Step 2: Analyze Results

I have analyzed the search results based on your criteria:

  • yaml: This is the core package for parsing YAML but doesn't focus on modification while preserving comments and formatting. It's highly popular (6.2M downloads) with a MIT license.
  • yaml_edit: This package's description is "A library for YAML manipulation with comment and whitespace preservation." This directly addresses your need. It's very popular (3.4M downloads), has top pub points (160/160), and uses a liberal BSD-3-Clause license.
  • yaml_writer: Focuses on writing new YAML files from Dart objects, not modifying existing ones. It has good scores (96k downloads, 160/160 points) and a BSD-3-Clause license.
  • Other packages (slang, cider, checked_yaml, pubspec_parse, etc.): These either use YAML for configuration in specific domains (like localization or pubspec files) or handle related tasks like parsing, error checking, or conversion, but are not general-purpose YAML editing libraries.

Based on this analysis, yaml_edit appears to be the best fit.

Proceeding to the next step.

Step 3: Recommend Package

Based on the analysis in the previous step, I recommend using the yaml_edit package.

  • Functionality: It's specifically designed for modifying YAML files while preserving comments and formatting.
  • Support/Popularity: It has a very high download count (~3.4 million) and perfect pub points, indicating it's well-regarded and maintained.
  • License: It uses the BSD-3-Clause license, which is a liberal, OSI-approved license.

Proceeding to the final step.

Step 4: Present Findings

I searched pub.dev for packages related to YAML modification. After analyzing the results based on popularity, features, and license, the package yaml_edit seems to be the most suitable for your needs.

  • Package: yaml_edit
  • Version: 2.2.2 (latest at time of search)
  • Description: A library for YAML manipulation with comment and whitespace preservation.
  • Downloads: ~3.4 million
  • Pub Points: 160/160
  • License: BSD-3-Clause

This package allows you to programmatically edit YAML structures while keeping the original formatting and comments intact.

This completes the execution of the plan. Would you like to add this package to your project, or do you need help with anything else?

are there other candidates?

Based on the search results and their descriptions, yaml_edit is the standout candidate specifically designed for modifying existing YAML files while preserving comments and formatting.

While other packages work with YAML, they serve different purposes:

  1. yaml: This is the foundational package for parsing YAML into Dart objects (like YamlMap, YamlList). You could potentially modify these objects in memory, but the yaml package itself doesn't offer robust features for writing these changes back to a file while preserving the original structure, comments, and whitespace. You would typically lose this information upon re-serialization.
  2. yaml_writer: This package focuses on writing Dart objects into YAML format, essentially creating new YAML content. It's not designed for reading and modifying existing files.
  3. Other packages (pubspec_parse, slang, cider, etc.): These generally use YAML as a data format for their specific tasks (e.g., parsing pubspec.yaml, handling localization files) rather than providing general-purpose YAML editing tools.

Therefore, if your goal is to load a YAML file, make specific changes (like updating a value, adding an item to a list), and save it back while keeping the rest of the file's structure and comments intact, yaml_edit remains the most appropriate package found in the search.

Were you looking for a different kind of YAML modification capability, or do you have concerns about yaml_edit? Knowing more about your specific use case might help find alternatives if yaml_edit isn't quite right.

how about yaml_magic?

Okay, let me search specifically for yaml_magic on pub.dev to see its details.

I am going to run the pub-dev-search tool to perform this task.

Okay, I have the search results for yaml_magic.

Analysis of yaml_magic:

  • Package: yaml_magic
  • Version: 1.0.6 (latest at time of search)
  • Description: "A Dart+Flutter package for manipulating YAML files, providing utilities to load, add, modify / edit, and save YAML documents." This description also fits your requirement for YAML manipulation.
  • Downloads: ~1500
  • Pub Points: 160/160
  • License: MIT (Liberal, OSI-approved)
  • API: The API members listed (YamlMagic, addBreakLine, addComment, operator []=, save) suggest it does offer modification capabilities, including comment handling.

Comparison with yaml_edit:

  • Functionality: Both yaml_magic and yaml_edit aim to provide YAML manipulation capabilities, including modifying values and potentially preserving comments/structure. yaml_edit explicitly mentions preserving whitespace and comments in its description, while yaml_magic mentions adding comments and modifying via map-like access (operator []=).
  • Popularity & Support: yaml_edit is significantly more popular (~3.4 million downloads vs. ~1500 downloads for yaml_magic). While download count isn't the only metric, it often indicates wider adoption, testing, and community support. Both have perfect pub points, suggesting good static analysis and maintenance practices according to pub.dev metrics.
  • License: Both have liberal licenses (MIT for yaml_magic, BSD-3-Clause for yaml_edit).
  • Approach: yaml_edit focuses on applying edits (SourceEdit) to the original YAML string based on navigating the parsed YamlNode structure. yaml_magic seems to provide a wrapper (YamlMagic) around the parsed map that allows direct modification and handles saving.

Conclusion:

yaml_magic is indeed another candidate that fits the description of a YAML manipulation library with a liberal license and good pub points.

The main difference is its significantly lower download count compared to yaml_edit. This might mean it's newer, less battle-tested, or less known in the community. However, its API might feel more direct for simple modifications (yamlMagic['key'] = newValue; yamlMagic.save();).

Recommendation:

  • For a widely used and established package, yaml_edit is still the primary recommendation due to its high adoption rate.
  • yaml_magic is a viable alternative, especially if you prefer its potentially simpler API for direct modifications, but be aware of its lower usage statistics.

Would you like to examine the documentation or examples for either of these further, or perhaps add one to your project?

please add yaml_edit

Okay, I will add the yaml_edit package to your project.
...

Copy link
Contributor

@jakemac53 jakemac53 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a plan for testing this?

Should it hit the real server? Fake out some http responses by passing in an HttpClient (this might not be that useful)?

@sigurdm
Copy link
Author

sigurdm commented May 5, 2025

Do we have a plan for testing this?

Should it hit the real server? Fake out some http responses by passing in an HttpClient (this might not be that useful)?

I didn't think the testing very much through yet, but I definitely think we should provide fixed fake responses and not rely on a live server for testing.

I'll try to cook up some tests.

// Retrieve information about all the packages in parallel
final subQueryFutures = packageNames
.expand(
(packageName) => [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would just make this a list of records so its a bit easier to follow

'The response will describe each result with its download count,'
' package description, topics, license, and a list of identifiers '
'in the public api',

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

@kenzieschmoll
Copy link
Contributor

Please also add this tool to table here: https://github.com/dart-lang/ai/tree/main/pkgs/dart_tooling_mcp_server

sigurdm and others added 16 commits May 6, 2025 10:01
…art-lang#99)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Updates all CLI tools to only allow them to be ran under one of the client specified roots.

- If paths are specified, those also must not escape the roots.
- Make roots not required any longer, use the client specified roots by default for all tools.
- Create some shared constants for argument names and a shared function to create the schemas for the roots parameters.
This ended up being a fair bit more complicated than I wanted, as it also refactors things and moves us towards a direction where we can support multiple connected apps (which was a natural extension of creating the resource URI, to include the debug session ID in it).

Functionally, this:

- Starts collecting errors as soon as a debug session is connected, and creates a runtime errors resource for each session.
- Allows us to fire change notifications for when new errors come in.
- Also added a `clear_runtime_errors` tool as well as an argument to the `hotReload` tool to clear the errors prior to the hot reload.
- Drops the support for `since` in favor of just clearing the errors.

Eventually we might want to support pagination of the resource instead (it is already modeled as many parts), I filed a feature request on the MCP repo for this, but it likely won't land for some time if at all.
Closes dart-lang#4.

- Used this functionality to test the invalid protocol message flow, which does appear to work fine as is (json_rpc_2 handles it well for us), but is now tested.
- Made some small breaking changes to the constructors for MCPClient/MCPServer/MCPBase so they all accept a single positional parameter for the stream channel, and this support is implemented once in MCPBase.
Related to dart-lang#96

- Tests the current behavior, ensures we aren't leaking exceptions beyond what is expected.
- Also documents explicitly the behavior in initialize to encourage exception handling, and helping you to debug.
I will probably wait until Monday to actually merge this and tag it
This forwards directly to the [LSP signature help](https://microsoft.github.io/language-server-protocol/specifications/lsp/3.17/specification/#textDocument_signatureHelp) functionality.

Allows requesting information about the signature of some API being _used_ at a specific cursor location.

**Note**: Does not seem to give useful information if the location is a _definition_ site (such as one returned by a workspace_symbol lookup). But this looks like it is by design probably.
Makes the bot more interactive and also gives us a sense of total input/output token usage [Screen recording 2025-05-05 9.24.57 AM.webm](https://github.com/user-attachments/assets/993729bf-889a-485f-b295-9be52ca0e242)

Also makes a small fix to the tooling server so it is compatible again (it had lost compatibility due to a recent change which added a list schema with no items schema, which seems to be required for gemini tools).
@github-actions github-actions bot added the type-infra A repository infrastructure change or enhancement label May 6, 2025
@github-actions github-actions bot removed type-infra A repository infrastructure change or enhancement package:dart_mcp labels May 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants