filesystem - load files into model context #307

snaggle-ai · 2024-12-11T10:40:18Z

Description

Adds resource templates and prompts to the filesystem reference server to load files and folders into model context.
Applied a limit of 30 files for loading into context.

loads files (e.g. images, pdfs) into context (rather than as base64 in a message).
demonstrates a practical use of prompts for loading files into context.

Note: This differs from drag/drop of files into Claude is that this allows the chat to keep track of the uris of files in the context and pass them to tools

Server Details

Server: filesystem
Changes to: resources, prompts

Motivation and Context

Allows loading images, pdf etc. into model context. There are lots of reasons why files would be loaded into context.
Tools do not load into model context but into a message.
Model context is preferable for images, pdfs and most non-text data
Model context is the only way to load larger files - these cannot be loaded via tools because they hit message response size limits.
demonstrates a practical use of prompts for loading files into context.

How Has This Been Tested?

Tested prompts with Claude

Breaking Changes

None

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation update

Checklist

I have read the MCP Protocol Documentation
My changes follows MCP security best practices
I have updated the server's README accordingly
I have tested this with an LLM client
My code follows the repository's style guidelines
New and existing tests pass locally
I have added appropriate error handling
I have documented all environment variables and configuration options

Additional context

Addresses this issue: #297
Relevant to this discussion: modelcontextprotocol/modelcontextprotocol#90
Very valuable conversation on reddit: https://www.reddit.com/r/mcp/comments/1hamhob/comment/m1cbq30/

snaggle-ai · 2024-12-11T10:42:55Z

src/filesystem/README.md


 **Note**: The server will only allow operations within directories specified via `args`.

 ## API

 ### Resources

- `file://system`: File system operations interface


this was/is not true, there are no resources exposed by the filesystem mcp before this PR

jspahrsummers

Thanks, this is awesome! Just a few questions/suggestions, then happy to merge this in.

jspahrsummers · 2024-12-16T14:22:56Z

src/filesystem/index.ts

+    // Handle both file:// and folder:// URIs
+    const match = request.params.uri.match(/^(file|folder):\/\/(.+)/);
+    if (!match) {
+      throw new Error('Invalid URI format');
+    }
+
+    const [, scheme, pathStr] = match;


Consider using the URL class for parsing here–makes it easier to check the scheme and extract different parts of the URL.

great point!

jspahrsummers · 2024-12-16T14:23:09Z

src/filesystem/index.ts

+    // Convert URI slashes to platform-specific path separators
+    const platformPath = pathStr.split('/').join(path.sep);


I think this shouldn't be needed—forward slashes work fine on Windows.

jspahrsummers · 2024-12-16T14:24:37Z

src/filesystem/index.ts

+  // If file is smaller than 100kb and type is unknown, treat as text
+  if (stats.size < 100 * 1024) {
+    return FileType.TEXT;
+  }
+
+  // Otherwise fall back to binary
+  return FileType.BINARY;


TIOLI: Best strategy here is probably to try reading it as a string, and if that fails, fall back to binary. This is something we could do later, though.

I think it will always be possible to "read" any file "as a string", just the string might be absolute garbage and weird characters.

I was thinking about some options around this

have separate read_binary, read_image, read_text functions - the caller will generally know what they want and could keep this approach as a read_auto or someting

alternatively make an optional input argument for binary/image/text and fall back to this behaviour if it's not specified

jspahrsummers · 2024-12-16T14:30:44Z

src/filesystem/index.ts

@@ -565,6 +573,137 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
        };
      }

+      case "load_folder": {


Did you mean to leave this tool in? I don't see it in the README.

Good catch thank you, I will remove it - I was using this personally but I think it's still a bit too janky to commit.

Many issues/questions:
Should it recurse?
Loading more than 5 files into context this way isn't supported by Claude so it frequently breaks

I can definitely see how this could blow out the context (depending on the folder) off the cuff I imagine a good ux would be:

don't recurse, but indicate that there are nested folder which an AI can further load

have some safeguards that prevent loading in too much data (maybe a text block saying there are "xyz more files" or something?)

jerome3o-anthropic

hey - I'm just getting spun up on this, these all look like great changes!

I haven't dug into the code for a proper review, but because I'm time poor atm, I'm keen to know if @snaggle-ai you're still keen to iterate on this/see it merged? I think they all look like awesome additions.

jerome3o-anthropic · 2025-02-07T00:28:30Z

src/filesystem/index.ts

@@ -565,6 +573,137 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
        };
      }

+      case "load_folder": {


I can definitely see how this could blow out the context (depending on the folder) off the cuff I imagine a good ux would be:

don't recurse, but indicate that there are nested folder which an AI can further load

have some safeguards that prevent loading in too much data (maybe a text block saying there are "xyz more files" or something?)

cliffhall · 2025-04-15T16:51:01Z

Hey @snaggle-ai are you still planning to amend this PR? It has happy nods from core team members, and I'm happy to test.

snaggle-ai added 4 commits December 11, 2024 11:20

updated readme

8712272

support for loading files into context

760cabf

fix imports and remove commented out code

0d0b3ff

update package-lock.json

d071a04

snaggle-ai commented Dec 11, 2024

View reviewed changes

jspahrsummers requested changes Dec 16, 2024

View reviewed changes

dsp-ant force-pushed the main branch 24 times, most recently from 93fb32f to 48e3d4a Compare January 13, 2025 20:34

dsp-ant force-pushed the main branch 25 times, most recently from c114833 to 4f3dc11 Compare January 14, 2025 03:05

jerome3o-anthropic reviewed Feb 7, 2025

View reviewed changes

olaservo added enhancement New feature or request server-filesystem Reference implementation for the Filesystem MCP server - src/filesystem labels Mar 25, 2025

cliffhall added the waiting for submitter Waiting for the submitter to provide more info label Apr 15, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

filesystem - load files into model context #307

filesystem - load files into model context #307

snaggle-ai commented Dec 11, 2024 •

edited

Loading

snaggle-ai Dec 11, 2024

jspahrsummers left a comment

jspahrsummers Dec 16, 2024

snaggle-ai Dec 17, 2024

jspahrsummers Dec 16, 2024

jspahrsummers Dec 16, 2024

snaggle-ai Dec 17, 2024

jspahrsummers Dec 16, 2024

snaggle-ai Dec 17, 2024

jerome3o-anthropic Feb 7, 2025

jerome3o-anthropic left a comment

jerome3o-anthropic Feb 7, 2025

cliffhall commented Apr 15, 2025

		// Convert URI slashes to platform-specific path separators
		const platformPath = pathStr.split('/').join(path.sep);

filesystem - load files into model context #307

Are you sure you want to change the base?

filesystem - load files into model context #307

Conversation

snaggle-ai commented Dec 11, 2024 • edited Loading

Description

Server Details

Motivation and Context

How Has This Been Tested?

Breaking Changes

Types of changes

Checklist

Additional context

Choose a reason for hiding this comment

jspahrsummers left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jerome3o-anthropic left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cliffhall commented Apr 15, 2025

snaggle-ai commented Dec 11, 2024 •

edited

Loading