Skip to content

Commit d75379e

Browse files
copyleftdevcopyleftdev
committed
fix: Server handling of JSON-RPC requests and improve project workflow
This commit includes several key improvements: 1. Fixed server JSON-RPC handling: - Properly handle jsoniter.RawMessage in request/response parsing - Ensure 200 OK status for all JSON-RPC responses per spec - Fixed tests to handle base64 encoded IDs correctly 2. Added comprehensive Makefile with targets for: - Building, testing, and running the server - Configuration validation and provider-config checks - Docker support and cross-platform builds - Special test modes (integration, live tests) 3. Updated README with: - Makefile usage instructions - Docker support documentation - Improved configuration and testing sections These changes ensure proper JSON-RPC protocol compliance and provide better developer workflow for building, testing and running the server.
0 parents  commit d75379e

File tree

4 files changed

+722
-0
lines changed

4 files changed

+722
-0
lines changed

‎Makefile

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# MCP Subfinder Server Makefile
2+
3+
# Go parameters
4+
GOCMD = go
5+
GOBUILD = $(GOCMD) build
6+
GOCLEAN = $(GOCMD) clean
7+
GOTEST = $(GOCMD) test
8+
GOGET = $(GOCMD) get
9+
GOMOD = $(GOCMD) mod
10+
GOLINT = golangci-lint
11+
BINARY_NAME = mcp-subfinder-server
12+
BINARY_UNIX = $(BINARY_NAME)_unix
13+
MAIN_PATH = .
14+
15+
# Build flags
16+
LDFLAGS = -ldflags "-s -w"
17+
18+
# Default port for the server
19+
PORT ?= 8080
20+
21+
# Default provider config location
22+
PROVIDER_CONFIG ?= provider-config.yaml
23+
24+
.PHONY: all build clean test coverage lint deps fmt help run tidy check-config integration-test live-test docker docker-run
25+
26+
# Default target
27+
all: test build
28+
29+
# Build the project
30+
build:
31+
@echo "Building..."
32+
$(GOBUILD) $(LDFLAGS) -o $(BINARY_NAME) $(MAIN_PATH)
33+
34+
# Build for Unix/Linux
35+
build-linux:
36+
@echo "Building for Linux..."
37+
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GOBUILD) $(LDFLAGS) -o $(BINARY_UNIX) $(MAIN_PATH)
38+
39+
# Clean the project
40+
clean:
41+
@echo "Cleaning..."
42+
$(GOCLEAN)
43+
rm -f $(BINARY_NAME)
44+
rm -f $(BINARY_UNIX)
45+
rm -f coverage.out
46+
rm -f coverage.html
47+
48+
# Run tests
49+
test:
50+
@echo "Running tests..."
51+
$(GOTEST) -v ./...
52+
53+
# Run tests with coverage
54+
coverage:
55+
@echo "Running tests with coverage..."
56+
$(GOTEST) -coverprofile=coverage.out ./...
57+
$(GOCMD) tool cover -html=coverage.out -o coverage.html
58+
@echo "Coverage report generated at coverage.html"
59+
60+
# Run integration tests
61+
integration-test:
62+
@echo "Running integration tests..."
63+
ENABLE_INTEGRATION_TESTS=1 $(GOTEST) -v ./...
64+
65+
# Run live subfinder tests
66+
live-test:
67+
@echo "Running live subfinder tests..."
68+
ENABLE_LIVE_TESTS=1 $(GOTEST) -v ./internal/subfinder/...
69+
70+
# Run linter
71+
lint:
72+
@echo "Running linter..."
73+
$(GOLINT) run
74+
75+
# Download dependencies
76+
deps:
77+
@echo "Downloading dependencies..."
78+
$(GOGET) -v ./...
79+
80+
# Format the code
81+
fmt:
82+
@echo "Formatting code..."
83+
$(GOCMD) fmt ./...
84+
85+
# Update go.mod and go.sum
86+
tidy:
87+
@echo "Tidying dependencies..."
88+
$(GOMOD) tidy
89+
90+
# Check if provider config exists
91+
check-config:
92+
@if [ ! -f $(PROVIDER_CONFIG) ]; then \
93+
echo "Provider config not found at $(PROVIDER_CONFIG)"; \
94+
echo "Using default sources only. For optimal results, add API keys."; \
95+
else \
96+
echo "Provider config found at $(PROVIDER_CONFIG)"; \
97+
fi
98+
99+
# Run the server
100+
run: check-config
101+
@echo "Running server on port $(PORT)..."
102+
$(GOCMD) run $(MAIN_PATH) -port $(PORT) -provider-config $(PROVIDER_CONFIG)
103+
104+
# Build a Docker image
105+
docker:
106+
@echo "Building Docker image..."
107+
docker build -t $(BINARY_NAME) .
108+
109+
# Run the server in Docker
110+
docker-run:
111+
@echo "Running server in Docker on port $(PORT)..."
112+
docker run -p $(PORT):$(PORT) -v $(PWD)/$(PROVIDER_CONFIG):/app/$(PROVIDER_CONFIG) $(BINARY_NAME) -port $(PORT) -provider-config /app/$(PROVIDER_CONFIG)
113+
114+
# Show help
115+
help:
116+
@echo "Make targets:"
117+
@echo " all - Run tests and build"
118+
@echo " build - Build the binary"
119+
@echo " build-linux - Build for Linux"
120+
@echo " clean - Remove binaries and coverage files"
121+
@echo " test - Run tests"
122+
@echo " integration-test - Run integration tests"
123+
@echo " live-test - Run live subfinder tests"
124+
@echo " coverage - Run tests with coverage report"
125+
@echo " lint - Run linter"
126+
@echo " deps - Download dependencies"
127+
@echo " fmt - Format the code"
128+
@echo " tidy - Update go.mod and go.sum"
129+
@echo " check-config - Check if provider config exists"
130+
@echo " run - Run the server (PORT=8080 by default)"
131+
@echo " docker - Build Docker image"
132+
@echo " docker-run - Run in Docker"
133+
@echo " help - Show this help message"
134+
@echo ""
135+
@echo "Variables:"
136+
@echo " PORT - Server port (default: 8080)"
137+
@echo " PROVIDER_CONFIG - Path to provider config file (default: provider-config.yaml)"

‎README.md

Lines changed: 246 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,246 @@
1+
# MCP Subfinder Server
2+
3+
[![License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
4+
[![Go Version](https://img.shields.io/badge/Go-1.20+-00ADD8.svg)](https://golang.org/)
5+
[![MCP Version](https://img.shields.io/badge/MCP-v0.3-success.svg)](https://github.com/copyleftdev/mcp-subfinder-server)
6+
[![ProjectDiscovery](https://img.shields.io/badge/Powered%20by-ProjectDiscovery-orange.svg)](https://github.com/projectdiscovery/subfinder)
7+
8+
A Model Context Protocol (MCP) server that wraps [ProjectDiscovery's subfinder](https://github.com/projectdiscovery/subfinder) tool for powerful subdomain enumeration through a JSON-RPC API.
9+
10+
## Architecture
11+
12+
```mermaid
13+
flowchart LR
14+
Client([Client]) -->|JSON-RPC| MCP[MCP Server]
15+
MCP -->|Initialize/Tools List| Client
16+
MCP -->|Handles Request| SF[Subfinder Wrapper]
17+
SF -->|Configuration| CFG[provider-config.yaml]
18+
SF -->|Calls| PD[ProjectDiscovery Subfinder]
19+
PD -->|Passive Sources| API1[Public & Private APIs]
20+
PD -->|Results| SF
21+
SF -->|Processed Results| MCP
22+
MCP -->|JSON Response| Client
23+
```
24+
25+
## Credit
26+
27+
All the heavy lifting for subdomain enumeration is done by [ProjectDiscovery's subfinder](https://github.com/projectdiscovery/subfinder). This project is simply a MCP server wrapper around their excellent tool.
28+
29+
## Overview
30+
31+
MCP Subfinder Server provides:
32+
33+
- JSON-RPC API to enumerate subdomains for a given domain
34+
- Support for recursive subdomain discovery
35+
- Source filtering capabilities
36+
- Configurable timeouts and threading
37+
- Detailed logging for troubleshooting
38+
39+
## Installation
40+
41+
```bash
42+
# Clone the repository
43+
git clone https://github.com/copyleftdev/mcp-subfinder-server.git
44+
cd mcp-subfinder-server
45+
46+
# Build the server using the Makefile
47+
make build
48+
```
49+
50+
## Usage
51+
52+
The server can be run using the Makefile, which provides several helpful commands:
53+
54+
```bash
55+
# Run the server on the default port (8080)
56+
make run
57+
58+
# Run the server on a custom port
59+
PORT=9090 make run
60+
61+
# Specify a different provider config file
62+
PROVIDER_CONFIG=my-custom-config.yaml make run
63+
```
64+
65+
### Available Makefile Commands
66+
67+
```bash
68+
# Show all available commands
69+
make help
70+
71+
# Run tests
72+
make test
73+
74+
# Run integration tests
75+
make integration-test
76+
77+
# Run live subfinder tests
78+
make live-test
79+
80+
# Generate test coverage report
81+
make coverage
82+
83+
# Format the code
84+
make fmt
85+
86+
# Build for Linux
87+
make build-linux
88+
89+
# Clean the project
90+
make clean
91+
```
92+
93+
## Configuration
94+
95+
For optimal results, add your API keys to the `provider-config.yaml` file. This allows subfinder to use premium sources for better subdomain discovery.
96+
97+
The provider-config.yaml file is checked automatically when running the server with `make run`.
98+
99+
## API Usage
100+
101+
The server exposes a JSON-RPC API at `http://localhost:8080/mcp`.
102+
103+
### Basic Usage Examples with curl
104+
105+
#### 1. Initialize Connection
106+
107+
```bash
108+
curl -X POST http://localhost:8080/mcp \
109+
-H "Content-Type: application/json" \
110+
-d '{
111+
"jsonrpc": "2.0",
112+
"id": 1,
113+
"method": "initialize",
114+
"params": {
115+
"protocolVersion": "0.3"
116+
}
117+
}'
118+
```
119+
120+
#### 2. List Available Tools
121+
122+
```bash
123+
curl -X POST http://localhost:8080/mcp \
124+
-H "Content-Type: application/json" \
125+
-d '{
126+
"jsonrpc": "2.0",
127+
"id": 2,
128+
"method": "tools.list"
129+
}'
130+
```
131+
132+
#### 3. Basic Subdomain Enumeration
133+
134+
```bash
135+
curl -X POST http://localhost:8080/mcp \
136+
-H "Content-Type: application/json" \
137+
-d '{
138+
"jsonrpc": "2.0",
139+
"id": 3,
140+
"method": "tools.call",
141+
"params": {
142+
"name": "enumerateSubdomains",
143+
"arguments": {
144+
"domain": "example.com"
145+
}
146+
}
147+
}'
148+
```
149+
150+
#### 4. Advanced Subdomain Enumeration
151+
152+
```bash
153+
curl -X POST http://localhost:8080/mcp \
154+
-H "Content-Type: application/json" \
155+
-d '{
156+
"jsonrpc": "2.0",
157+
"id": 4,
158+
"method": "tools.call",
159+
"params": {
160+
"name": "enumerateSubdomains",
161+
"arguments": {
162+
"domain": "example.com",
163+
"timeout": 120,
164+
"recursive": true,
165+
"maxDepth": 2,
166+
"sourcesFilter": "github,dnsdumpster,alienvault"
167+
}
168+
}
169+
}'
170+
```
171+
172+
#### 5. Enumeration with Source Exclusion
173+
174+
```bash
175+
curl -X POST http://localhost:8080/mcp \
176+
-H "Content-Type: application/json" \
177+
-d '{
178+
"jsonrpc": "2.0",
179+
"id": 5,
180+
"method": "tools.call",
181+
"params": {
182+
"name": "enumerateSubdomains",
183+
"arguments": {
184+
"domain": "example.com",
185+
"timeout": 60,
186+
"excludeSourcesFilter": "waybackarchive,threatcrowd"
187+
}
188+
}
189+
}'
190+
```
191+
192+
#### 6. Health Check
193+
194+
```bash
195+
curl -X GET http://localhost:8080/health
196+
```
197+
198+
## Available Options
199+
200+
When calling the `enumerateSubdomains` tool, the following options are available:
201+
202+
| Option | Type | Description | Default |
203+
|--------|------|-------------|---------|
204+
| domain | string | The domain to enumerate subdomains for (required) | - |
205+
| timeout | int | Timeout in seconds for the enumeration process | 120 |
206+
| recursive | bool | Whether to recursively check discovered subdomains | false |
207+
| maxDepth | int | Maximum depth for recursive enumeration | 2 |
208+
| sourcesFilter | string | Comma-separated list of sources to use | - |
209+
| excludeSourcesFilter | string | Comma-separated list of sources to exclude | - |
210+
211+
## Docker Support
212+
213+
The project includes Docker support through the Makefile:
214+
215+
```bash
216+
# Build a Docker image
217+
make docker
218+
219+
# Run the server in Docker
220+
make docker-run
221+
222+
# Run with custom port
223+
PORT=9090 make docker-run
224+
```
225+
226+
## Testing
227+
228+
Run tests using the Makefile:
229+
230+
```bash
231+
# Run all tests
232+
make test
233+
234+
# Run with test coverage
235+
make coverage
236+
```
237+
238+
A Postman collection is included in the `docs` folder for easy testing of all API endpoints.
239+
240+
## License
241+
242+
This project is licensed under the MIT License - see the LICENSE file for details.
243+
244+
## Author
245+
246+
[copyleftdev](https://github.com/copyleftdev)

0 commit comments

Comments
 (0)