Skip to content

Commit 42af216

Browse files
aminghadersohiaminghadersohi
committed
feat: Add MCP (Model Context Protocol) service scaffold for Apache Superset
This commit introduces the basic scaffold for the MCP service that enables AI agents to interact with Apache Superset programmatically. ## Key Components ### Core Structure - **CLI Interface**: `superset mcp run` and `superset mcp setup` commands - **FastMCP Server**: HTTP-based MCP server using FastMCP protocol - **Flask Integration**: Singleton pattern for Superset app integration - **Configuration Setup**: Automated setup script for MCP service config ### Files Added - `superset/cli/mcp.py`: CLI commands for running and setting up MCP service - `superset/mcp_service/`: Core MCP service package with server, app, and config - `superset/mcp_service/scripts/setup.py`: Configuration setup utilities - `superset/config_mcp.py`: MCP-specific configuration template - Development container configurations for MCP-enabled development ### Features - Basic HTTP transport for MCP communication - Configuration management for Superset integration - Development environment setup with Docker support - Node.js proxy support for alternative connection methods This scaffold provides the foundation for subsequent PRs that will add: - Authentication and authorization - Tool implementations (dashboards, charts, datasets) - Advanced middleware and permissions - Production-ready features The implementation follows Superset's architectural patterns and can be extended incrementally without breaking changes.
1 parent 05c6a1b commit 42af216

24 files changed

+1455
-138
lines changed

‎.devcontainer/README.md‎

Lines changed: 0 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,3 @@
33
For complete documentation on using GitHub Codespaces with Apache Superset, please see:
44

55
**[Setting up a Development Environment - GitHub Codespaces](https://superset.apache.org/docs/contributing/development#github-codespaces-cloud-development)**
6-
7-
## Pre-installed Development Environment
8-
9-
When you create a new Codespace from this repository, it automatically:
10-
11-
1. **Creates a Python virtual environment** using `uv venv`
12-
2. **Installs all development dependencies** via `uv pip install -r requirements/development.txt`
13-
3. **Sets up pre-commit hooks** with `pre-commit install`
14-
4. **Activates the virtual environment** automatically in all terminals
15-
16-
The virtual environment is located at `/workspaces/{repository-name}/.venv` and is automatically activated through environment variables set in the devcontainer configuration.

‎.devcontainer/default/devcontainer.json‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
// Extend the base configuration
3+
"extends": "../devcontainer-base.json",
4+
5+
"name": "Apache Superset Development (Default)",
6+
7+
// Forward ports for development
8+
"forwardPorts": [9001],
9+
"portsAttributes": {
10+
"9001": {
11+
"label": "Superset (via Webpack Dev Server)",
12+
"onAutoForward": "notify",
13+
"visibility": "public"
14+
}
15+
},
16+
17+
// Auto-start Superset on Codespace resume
18+
"postStartCommand": ".devcontainer/start-superset.sh"
19+
}

‎.devcontainer/devcontainer-base.json‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
{
2+
"name": "Apache Superset Development",
3+
// Keep this in sync with the base image in Dockerfile (ARG PY_VER)
4+
// Using the same base as Dockerfile, but non-slim for dev tools
5+
"image": "python:3.11.13-bookworm",
6+
7+
"features": {
8+
"ghcr.io/devcontainers/features/docker-in-docker:2": {
9+
"moby": true,
10+
"dockerDashComposeVersion": "v2"
11+
},
12+
"ghcr.io/devcontainers/features/node:1": {
13+
"version": "20"
14+
},
15+
"ghcr.io/devcontainers/features/git:1": {},
16+
"ghcr.io/devcontainers/features/common-utils:2": {
17+
"configureZshAsDefaultShell": true
18+
},
19+
"ghcr.io/devcontainers/features/sshd:1": {
20+
"version": "latest"
21+
}
22+
},
23+
24+
// Run commands after container is created
25+
"postCreateCommand": "chmod +x .devcontainer/setup-dev.sh && .devcontainer/setup-dev.sh",
26+
27+
// VS Code customizations
28+
"customizations": {
29+
"vscode": {
30+
"extensions": [
31+
"ms-python.python",
32+
"ms-python.vscode-pylance",
33+
"charliermarsh.ruff",
34+
"dbaeumer.vscode-eslint",
35+
"esbenp.prettier-vscode"
36+
]
37+
}
38+
}
39+
}

‎.devcontainer/setup-dev.sh‎

Lines changed: 19 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -3,76 +3,30 @@
33

44
echo "🔧 Setting up Superset development environment..."
55

6-
# System dependencies and uv are now pre-installed in the Docker image
7-
# This speeds up Codespace creation significantly!
8-
9-
# Create virtual environment using uv
10-
echo "🐍 Creating Python virtual environment..."
11-
if ! uv venv; then
12-
echo "❌ Failed to create virtual environment"
13-
exit 1
14-
fi
15-
16-
# Install Python dependencies
17-
echo "📦 Installing Python dependencies..."
18-
if ! uv pip install -r requirements/development.txt; then
19-
echo "❌ Failed to install Python dependencies"
20-
echo "💡 You may need to run this manually after the Codespace starts"
21-
exit 1
22-
fi
23-
24-
# Install pre-commit hooks
25-
echo "🪝 Installing pre-commit hooks..."
26-
if source .venv/bin/activate && pre-commit install; then
27-
echo "✅ Pre-commit hooks installed"
28-
else
29-
echo "⚠️ Pre-commit hooks installation failed (non-critical)"
30-
fi
6+
# The universal image has most tools, just need Superset-specific libs
7+
echo "📦 Installing Superset-specific dependencies..."
8+
sudo apt-get update
9+
sudo apt-get install -y \
10+
libsasl2-dev \
11+
libldap2-dev \
12+
libpq-dev \
13+
tmux \
14+
gh
15+
16+
# Install uv for fast Python package management
17+
echo "📦 Installing uv..."
18+
curl -LsSf https://astral.sh/uv/install.sh | sh
19+
20+
# Add cargo/bin to PATH for uv
21+
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.bashrc
22+
echo 'export PATH="$HOME/.cargo/bin:$PATH"' >> ~/.zshrc
3123

3224
# Install Claude Code CLI via npm
3325
echo "🤖 Installing Claude Code..."
34-
if npm install -g @anthropic-ai/claude-code; then
35-
echo "✅ Claude Code installed"
36-
else
37-
echo "⚠️ Claude Code installation failed (non-critical)"
38-
fi
26+
npm install -g @anthropic-ai/claude-code
3927

4028
# Make the start script executable
4129
chmod +x .devcontainer/start-superset.sh
4230

43-
# Add bashrc additions for automatic venv activation
44-
echo "🔧 Setting up automatic environment activation..."
45-
if [ -f ~/.bashrc ]; then
46-
# Check if we've already added our additions
47-
if ! grep -q "Superset Codespaces environment setup" ~/.bashrc; then
48-
echo "" >> ~/.bashrc
49-
cat .devcontainer/bashrc-additions >> ~/.bashrc
50-
echo "✅ Added automatic venv activation to ~/.bashrc"
51-
else
52-
echo "✅ Bashrc additions already present"
53-
fi
54-
else
55-
# Create bashrc if it doesn't exist
56-
cat .devcontainer/bashrc-additions > ~/.bashrc
57-
echo "✅ Created ~/.bashrc with automatic venv activation"
58-
fi
59-
60-
# Also add to zshrc since that's the default shell
61-
if [ -f ~/.zshrc ] || [ -n "$ZSH_VERSION" ]; then
62-
if ! grep -q "Superset Codespaces environment setup" ~/.zshrc; then
63-
echo "" >> ~/.zshrc
64-
cat .devcontainer/bashrc-additions >> ~/.zshrc
65-
echo "✅ Added automatic venv activation to ~/.zshrc"
66-
fi
67-
fi
68-
6931
echo "✅ Development environment setup complete!"
70-
echo ""
71-
echo "📝 The virtual environment will be automatically activated in new terminals"
72-
echo ""
73-
echo "🔄 To activate in this terminal, run:"
74-
echo " source ~/.bashrc"
75-
echo ""
76-
echo "🚀 To start Superset:"
77-
echo " start-superset"
78-
echo ""
32+
echo "🚀 Run '.devcontainer/start-superset.sh' to start Superset"

‎.devcontainer/start-superset.sh‎

Lines changed: 22 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
#!/bin/bash
22
# Startup script for Superset in Codespaces
33

4-
# Log to a file for debugging
5-
LOG_FILE="/tmp/superset-startup.log"
6-
echo "[$(date)] Starting Superset startup script" >> "$LOG_FILE"
7-
echo "[$(date)] User: $(whoami), PWD: $(pwd)" >> "$LOG_FILE"
8-
94
echo "🚀 Starting Superset in Codespaces..."
105
echo "🌐 Frontend will be available at port 9001"
116

7+
# Check if MCP is enabled
8+
if [ "$ENABLE_MCP" = "true" ]; then
9+
echo "🤖 MCP Service will be available at port 5008"
10+
fi
11+
1212
# Find the workspace directory (Codespaces clones as 'superset', not 'superset-2')
1313
WORKSPACE_DIR=$(find /workspaces -maxdepth 1 -name "superset*" -type d | head -1)
1414
if [ -n "$WORKSPACE_DIR" ]; then
@@ -18,71 +18,32 @@ else
1818
echo "📁 Using current directory: $(pwd)"
1919
fi
2020

21-
# Wait for Docker to be available
22-
echo "⏳ Waiting for Docker to start..."
23-
echo "[$(date)] Waiting for Docker..." >> "$LOG_FILE"
24-
max_attempts=30
25-
attempt=0
26-
while ! docker info > /dev/null 2>&1; do
27-
if [ $attempt -eq $max_attempts ]; then
28-
echo "❌ Docker failed to start after $max_attempts attempts"
29-
echo "[$(date)] Docker failed to start after $max_attempts attempts" >> "$LOG_FILE"
30-
echo "🔄 Please restart the Codespace or run this script manually later"
31-
exit 1
32-
fi
33-
echo " Attempt $((attempt + 1))/$max_attempts..."
34-
echo "[$(date)] Docker check attempt $((attempt + 1))/$max_attempts" >> "$LOG_FILE"
35-
sleep 2
36-
attempt=$((attempt + 1))
37-
done
38-
echo "✅ Docker is ready!"
39-
echo "[$(date)] Docker is ready" >> "$LOG_FILE"
40-
41-
# Check if Superset containers are already running
42-
if docker ps | grep -q "superset"; then
43-
echo "✅ Superset containers are already running!"
44-
echo ""
45-
echo "🌐 To access Superset:"
46-
echo " 1. Click the 'Ports' tab at the bottom of VS Code"
47-
echo " 2. Find port 9001 and click the globe icon to open"
48-
echo " 3. Wait 10-20 minutes for initial startup"
49-
echo ""
50-
echo "📝 Login credentials: admin/admin"
51-
exit 0
21+
# Check if docker is running
22+
if ! docker info > /dev/null 2>&1; then
23+
echo "⏳ Waiting for Docker to start..."
24+
sleep 5
5225
fi
5326

5427
# Clean up any existing containers
5528
echo "🧹 Cleaning up existing containers..."
56-
docker-compose -f docker-compose-light.yml down
29+
docker-compose -f docker-compose-light.yml --profile mcp down
5730

5831
# Start services
59-
echo "🏗️ Starting Superset in background (daemon mode)..."
32+
echo "🏗️ Building and starting services..."
6033
echo ""
61-
62-
# Start in detached mode
63-
docker-compose -f docker-compose-light.yml up -d
64-
65-
echo ""
66-
echo "✅ Docker Compose started successfully!"
67-
echo ""
68-
echo "📋 Important information:"
69-
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
70-
echo "⏱️ Initial startup takes 10-20 minutes"
71-
echo "🌐 Check the 'Ports' tab for your Superset URL (port 9001)"
72-
echo "👤 Login: admin / admin"
34+
echo "📝 Once started, login with:"
35+
echo " Username: admin"
36+
echo " Password: admin"
7337
echo ""
74-
echo "📊 Useful commands:"
75-
echo " docker-compose -f docker-compose-light.yml logs -f # Follow logs"
76-
echo " docker-compose -f docker-compose-light.yml ps # Check status"
77-
echo " docker-compose -f docker-compose-light.yml down # Stop services"
78-
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
79-
echo ""
80-
echo "💤 Keeping terminal open for 60 seconds to test persistence..."
81-
sleep 60
82-
echo "✅ Test complete - check if this terminal is still visible!"
38+
echo "📋 Running in foreground with live logs (Ctrl+C to stop)..."
8339

84-
# Show final status
85-
docker-compose -f docker-compose-light.yml ps
40+
# Run docker-compose and capture exit code
41+
if [ "$ENABLE_MCP" = "true" ]; then
42+
echo "🤖 Starting with MCP Service enabled..."
43+
docker-compose -f docker-compose-light.yml --profile mcp up
44+
else
45+
docker-compose -f docker-compose-light.yml up
46+
fi
8647
EXIT_CODE=$?
8748

8849
# If it failed, provide helpful instructions

‎.devcontainer/with-mcp/devcontainer.json‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
{
2+
// Extend the base configuration
3+
"extends": "../devcontainer-base.json",
4+
5+
"name": "Apache Superset Development with MCP",
6+
7+
// Forward ports for development
8+
"forwardPorts": [9001, 5008],
9+
"portsAttributes": {
10+
"9001": {
11+
"label": "Superset (via Webpack Dev Server)",
12+
"onAutoForward": "notify",
13+
"visibility": "public"
14+
},
15+
"5008": {
16+
"label": "MCP Service (Model Context Protocol)",
17+
"onAutoForward": "notify",
18+
"visibility": "private"
19+
}
20+
},
21+
22+
// Auto-start Superset with MCP on Codespace resume
23+
"postStartCommand": "ENABLE_MCP=true .devcontainer/start-superset.sh",
24+
25+
// Environment variables
26+
"containerEnv": {
27+
"ENABLE_MCP": "true"
28+
}
29+
}

‎docker/docker-bootstrap.sh‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,10 @@ case "${1}" in
7878
echo "Starting web app..."
7979
/usr/bin/run-server.sh
8080
;;
81+
mcp)
82+
echo "Starting MCP service..."
83+
superset mcp run --host 0.0.0.0 --port ${MCP_PORT:-5008} --debug
84+
;;
8185
*)
8286
echo "Unknown Operation!!!"
8387
;;

‎requirements/development.in‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,5 +16,5 @@
1616
# specific language governing permissions and limitations
1717
# under the License.
1818
#
19-
-e .[development,bigquery,druid,duckdb,gevent,gsheets,mysql,postgres,presto,prophet,trino,thumbnails]
19+
-e .[development,bigquery,druid,duckdb,fastmcp,gevent,gsheets,mysql,postgres,presto,prophet,trino,thumbnails]
2020
-e ./superset-extensions-cli[test]

‎superset/cli/mcp.py‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
"""CLI module for MCP service"""
18+
19+
import click
20+
from flask.cli import with_appcontext
21+
22+
from superset.mcp_service.scripts.setup import run_setup
23+
from superset.mcp_service.server import run_server
24+
25+
26+
@click.group()
27+
def mcp() -> None:
28+
"""Model Context Protocol service commands"""
29+
pass
30+
31+
32+
@mcp.command()
33+
@click.option("--host", default="127.0.0.1", help="Host to bind to")
34+
@click.option("--port", default=5008, help="Port to bind to")
35+
@click.option("--debug", is_flag=True, help="Enable debug mode")
36+
def run(host: str, port: int, debug: bool) -> None:
37+
"""Run the MCP service"""
38+
run_server(host=host, port=port, debug=debug)
39+
40+
41+
@mcp.command()
42+
@click.option("--force", is_flag=True, help="Force setup even if configuration exists")
43+
@with_appcontext
44+
def setup(force: bool) -> None:
45+
"""Set up MCP service for Apache Superset"""
46+
run_setup(force)

0 commit comments

Comments
 (0)