The IGG (Idea Generator Generator) project generates creative ideas using Markov chains trained on CSV data. It provides both a static web interface and an MCP (Model Context Protocol) server for integration with AI tools like Claude Code.

• 🔄 Automatic CSV → Markov Model Processing: Upload CSV files and get trained models
• 🌐 Static Web Interface: Browse and generate ideas from models
• 🔌 MCP Server Integration: Use with Claude Code and other AI tools
• ☁️ AWS CDK Infrastructure: Fully managed serverless deployment
• 📊 Model Caching: Efficient local and remote model storage

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Web Frontend  │    │   MCP Server    │    │ AWS CDK Stack   │
│                 │    │                 │    │                 │
│ - Static Site   │    │ - API Gateway   │    │ - S3 Buckets    │
│ - Model Browse  │    │ - Basic Auth    │    │ - API Gateway   │
│ - Idea Gen      │    │ - Lambda Funcs  │    │ - Lambda Funcs  │
│                 │    │ - Custom Domain │    │ - SSL Certs     │
└─────────────────┘    └─────────────────┘    └─────────────────┘

├── src/                                  # Python source code
│   ├── mcp_server.py                     # MCP server entry point
│   ├── mcp_markov_models.py              # MCP Markov logic  
│   ├── model_processor.py                # Lambda CSV processor
│   └── generate_markov_models.py         # Standalone utility
├── cdk/                                  # AWS CDK infrastructure
│   ├── app.py                            # CDK application
│   ├── custom_constructs/                # Reusable CDK components
│   │   ├── mcp_server_construct.py       # MCP server infrastructure
│   │   ├── static_site_construct.py      # Static site infrastructure
│   │   └── model_processor_construct.py  # Lambda processor infrastructure
│   └── stacks/                           # CDK stack definitions
│       ├── mcp_stack.py                  # MCP server infrastructure
│       └── static_site_stack.py          # Static site infrastructure
├── test/                                 # Unit tests
├── web/                                  # Frontend web interface
│   ├── index.html                        # Static site frontend
│   ├── samples/                          # Sample data files
│   └── script/                           # Frontend JavaScript
├── models/cache/                         # Local model cache
├── lambda-layer/                         # Lambda dependencies layer
└── layerator.py                          # Lambda layer builder script

• Python 3.12+ with Pipenv
• Node.js and AWS CDK CLI (for infrastructure)
• AWS CLI configured (for deployment)

# Clone and install dependencies
git clone <repository-url>
cd igg
pipenv install

# Run MCP server locally
pipenv run python src/mcp_server.py

After deploying the CDK stack, configure your MCP client using the deployed API Gateway endpoint:

{
  "mcpServers": {
    "igg-markov": {
      "url": "https://mcp.yourdomain.com/",
      "headers": {
        "Authorization": "Basic <base64-encoded-credentials>"
      }
    }
  }
}

Getting the MCP endpoint URL and credentials:

# Deploy and get outputs
cd cdk && pipenv run cdk deploy IggMcpStack

# The deployment will output:
# - McpApiGatewayUrl: Direct API Gateway URL (works immediately)
# - McpCustomDomainUrl: Custom domain URL (requires DNS setup)
# - McpAuthSecretArn: Secret ARN for credentials

# Get credentials from AWS Secrets Manager
aws secretsmanager get-secret-value \
  --secret-id <McpAuthSecretArn-from-output> \
  --query SecretString --output text

URL Options:

• Quick setup: Use McpApiGatewayUrl directly (no DNS required)
• Custom domain: Use McpCustomDomainUrl after setting up DNS CNAME record

# Build Lambda layer with heavy dependencies (pandas, nltk)
pipenv run python layerator.py

# Configure domains in config.json
cd cdk
cp config.json.example config.json
# Edit config.json with your domains and certificates

# Deploy infrastructure
pipenv run cdk deploy --all

# Set up DNS records as shown in deployment outputs

{
  "mcp": {
    "domain": "mcp.yourdomain.com",
    "certificateDomain": "mcp.yourdomain.com"
  },
  "static_site": {
    "domain": "static.yourdomain.com", 
    "certificate_arn": "arn:aws:acm:...",
    "bucket_name": "your-bucket-name"
  }
}

• IGG_BASE_URL: Override default model endpoint for MCP server
• BUCKET_NAME: S3 bucket name (set by CDK)
• INDEX_FILE: Index file name (default: index.json)

{
  "tool": "list_models"
}

{
  "tool": "generate_ideas",
  "arguments": {
    "model_name": "samples/sample.json",
    "count": 5
  }
}

{
  "tool": "generate_with_template",
  "arguments": {
    "model_name": "samples/sample.json",
    "template": "A $1 solution for $2 professionals",
    "count": 3
  }
}

1. Upload CSV: Place CSV file in S3 bucket (any path structure)
2. Auto-Processing: Lambda automatically converts CSV → Markov JSON
3. Index Update: index.json updated with model metadata
4. Access Models: Available via API Gateway and MCP server

Run the complete test suite:

pipenv run pytest test/ -v

Test specific modules:

pipenv run pytest test/test_model_processor.py -v
pipenv run pytest test/test_generate_markov_models.py -v

1. Upload CSV data to S3 bucket
2. Models automatically generated and indexed
3. Available immediately via MCP and web interface

The model processor uses a Lambda layer for heavy dependencies (pandas, nltk, numpy). The layer is built for Python 3.12:

# Rebuild layer when dependencies change
pipenv run python layerator.py

# Deploy updated layer
pipenv run cdk deploy IggStaticSiteStack

1. Add new methods to mcp_markov_models.py
2. Register tools in mcp_server.py
3. Add corresponding tests

1. Modify constructs in cdk/custom_constructs/
2. Update stack definitions in cdk/stacks/
3. Test with pipenv run cdk diff
4. Deploy with pipenv run cdk deploy

• Lambda function logs: /aws/lambda/ModelProcessor
• API Gateway logs: Available in CloudWatch

# Test model processor locally
pipenv run python -c "
import model_processor
result = model_processor.process_csv('col1,col2\nval1,val2', 'test.csv')
print(result)
"

• All AWS resources use minimal IAM permissions
• API Gateway secured with custom authorizers (MCP stack)
• HTTPS/TLS 1.2+ enforced on all endpoints
• S3 bucket access restricted to API Gateway
• Authentication secrets managed via AWS Secrets Manager

# Development
pipenv run cdk deploy --context environment=dev

# Production  
pipenv run cdk deploy --context environment=prod

# Validate infrastructure
pipenv run cdk synth
pipenv run cdk diff

# Deploy specific stacks
pipenv run cdk deploy IggMcpStack
pipenv run cdk deploy IggStaticSiteStack

Remove all AWS resources:

pipenv run cdk destroy --all

This project is licensed under the Apache License 2.0. See the LICENSE file for details.

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Run tests: pipenv run pytest
4. Commit changes: git commit -m 'Add amazing feature'
5. Push to branch: git push origin feature/amazing-feature
6. Open a Pull Request

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: See individual component READMEs in subdirectories