data-pipeline MCP Server

A comprehensive data engineering pipeline MCP server with 22 tools for data quality assessment, cleaning, feature engineering, sampling, aggregation, and synthetic data generation.

Built with mcp.server.fastmcp and pure Python standard library (no pandas/numpy).

Quick Start

# Install mcp
pip install mcp

# Run the server
python main.py

# Or with the MCP CLI for testing
python -c "from mcp.server.fastmcp import FastMCP; print('MCP server ready')"

Tools Overview

Data Quality (4 tools)

| Tool | Description | |---|---| | data_profiler | Data profiling: type inference, missing rate, uniqueness, descriptive statistics (mean/median/std/skew/kurtosis/quartiles) | | quality_scorer | 5-dimension quality scoring: completeness, uniqueness, consistency, accuracy, timeliness (weighted 0-100) | | outlier_detector | Outlier detection via IQR rule (Q1-1.5IQR, Q3+1.5IQR), Z-score (>3 sigma), and MAD-modified Z-score | | duplicate_finder | Exact + fuzzy duplicate detection using Levenshtein distance with configurable similarity threshold |

Data Cleaning (4 tools)

| Tool | Description | |---|---| | missing_imputer | Missing value imputation: mean, median, mode, forward-fill, backward-fill, linear interpolation | | normalizer | 5 normalization methods: Min-Max, Z-Score, Robust (median+IQR), Decimal Scaling, Log transform | | data_smoother | Time-series smoothing: moving average, exponential smoothing (alpha), median filter | | discretizer | Discretization: equal-width binning, equal-frequency binning, custom bins, one-hot encoding |

Feature Engineering (4 tools)

| Tool | Description | |---|---| | feature_selector | Feature selection: variance threshold, correlation threshold, mutual information estimate, recursive elimination (RFE proxy) | | pca_decomposer | PCA: centering → covariance matrix → eigen-decomposition (power iteration) → explained variance ratio | | scaler | Scaling: MinMax, Robust, MaxAbs, UnitLength (L1/L2/Inf norm) | | interaction_generator | Interaction features: pairwise product, ratio, polynomial expansion (degree 2/3) |

Sampling & Splitting (4 tools)

| Tool | Description | |---|---| | sampler | Sampling: random, systematic, stratified, bootstrap, reservoir sampling | | stratified_splitter | Stratified train/validation/test split preserving class proportions | | cross_validator | Cross-validation: K-Fold, Stratified K-Fold, Leave-Sequence-Out (LSOO), time-series rolling window | | data_balancer | Class balancing: oversampling, undersampling, SMOTE (linear interpolation synthesis) |

Aggregation & Transformation (4 tools)

| Tool | Description | |---|---| | pivot_table | Pivot table: group aggregation (count/sum/mean/median/std) with multi-level index | | data_aggregator | Multi-function aggregation: GroupBy with count/sum/mean/median/std/min/max/var | | column_encoder | Categorical encoding: Label, Ordinal, Frequency, Target encoding | | bin_counter | Bin statistics: count/mean/proportion per bin with ASCII bar chart visualization |

Data Generation (2 tools)

| Tool | Description | |---|---| | data_generator | Synthetic data generation: normal/uniform/categorical/time-series/integer/boolean + missing value injection | | sql_query_builder | Visual SQL query builder: SELECT/FROM/WHERE/GROUP BY/ORDER BY/JOIN/LIMIT with ASCII dependency graph |

Usage Examples

Data Profiling

{
  "data": [
    {"name": "Alice", "age": 30, "salary": 70000},
    {"name": "Bob", "age": 25, "salary": 55000},
    {"name": "Charlie", "age": 35, "salary": 90000}
  ]
}

Outlier Detection (IQR method)

{
  "data": [{"val": 10}, {"val": 12}, {"val": 11}, {"val": 100}],
  "column": "val",
  "method": "iqr"
}

Synthetic Data Generation

{
  "schema": {
    "id": {"type": "integer", "low": 1, "high": 1000},
    "score": {"type": "normal", "mean": 75, "std": 10},
    "grade": {"type": "categorical", "categories": ["A","B","C","D"], "probs": [0.2, 0.4, 0.3, 0.1]},
    "date": {"type": "timeseries", "start": "2024-01-01", "freq": "D"}
  },
  "n_rows": 30,
  "missing_rate": 0.05,
  "seed": 42
}

Data Format

All tools accept JSON input and return Markdown-formatted results. The data field supports two formats:

1. Record-oriented (preferred): [{"col": val, ...}, ...]
2. Row-oriented: [[val, ...], ...] with an accompanying "columns": ["col", ...] field

Requirements

• Python 3.10+
• mcp package (pip install mcp)

No other third-party dependencies required.