Met Museum Paintings Semantic Search

Search ~2,300 paintings from the Met Museum Open Access collection using AI-powered semantic search.

Keywords: art search, museum, paintings, Met Museum, semantic search, artwork discovery, visual similarity, art history

When to Use This Skill

Use this skill when users:

• Ask to find artworks by description ("find paintings of people on bridges")
• Want to explore museum collections
• Ask about specific artworks or artists at the Met
• Want to find visually similar artworks
• Upload an image to find matching paintings

API Base URL

https://museum-semantic-search.vercel.app/api/mcp

API Endpoints

POST /search - Search Artworks

Search the collection using text queries with optional filters.

curl -X POST https://museum-semantic-search.vercel.app/api/mcp/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "person with a dog",
    "mode": "semantic",
    "limit": 10
  }'

Request body:

{
  "query": "string (required)",
  "mode": "hybrid | semantic | keyword",
  "detail": "minimal | standard | full",
  "filters": {
    "artistName": "string (fuzzy matched)",
    "yearStart": 1800,
    "yearEnd": 1900,
    "department": "string (exact match)",
    "culture": "string (exact match)",
    "classification": "string (exact match)",
    "tags": ["string (exact match)"],
    "medium": "string (fuzzy matched)",
    "country": "string (exact match)",
    "onView": true,
    "isPublicDomain": true
  },
  "limit": 10,
  "offset": 0
}

Mode selection:

• "hybrid" (default, recommended): Combines keyword + semantic search. Best for most queries.
• "semantic": For purely conceptual/descriptive queries like "lonely figure in nature", "woman looking in mirror", "stormy seascape"
• "keyword": For exact matches like artist names or specific titles

Detail levels:

• "minimal": Just id, score, title, artist, date (fastest)
• "standard" (default): + medium, classification, department, tags, culture, image_url, source_url, alt_text (concise AI description)
• "full": + long_description (detailed AI description), similar_artworks, dimensions, etc. Use when you need to verify semantic content.

Response:

{
  "meta": {
    "query": "person with a dog",
    "mode": "hybrid",
    "total": 150,
    "returned": 10,
    "offset": 0,
    "limit": 10,
    "has_more": true,
    "filters_applied": {},
    "processing_time_ms": 234
  },
  "results": [
    {
      "id": "met_436524",
      "score": 0.85,
      "title": "A Boy with a Flying Squirrel",
      "artist": "John Singleton Copley",
      "date": "1765",
      "medium": "Oil on canvas",
      "classification": "Paintings",
      "department": "American Wing",
      "image_url": "https://images.metmuseum.org/...",
      "thumbnail_url": "https://images.metmuseum.org/.../small/...",
      "tags": ["Boys", "Portraits", "Squirrels"],
      "culture": "American",
      "source_url": "https://www.metmuseum.org/art/collection/search/436524",
      "alt_text": "Portrait of a young boy in blue satin holding a gold chain attached to a pet flying squirrel on a polished table"
    }
  ]
}

GET /artwork/{id} - Get Artwork Details

Get complete details for a specific artwork.

curl https://museum-semantic-search.vercel.app/api/mcp/artwork/met_436524

Response includes:

• Basic info: title, titles (alternate), artist, artists (with roles, bio, dates)
• Dates: date, date_start, date_end
• Physical: medium, dimensions
• Classification: classification, department, object_name
• Context: culture, period, dynasty, country, region
• Collection: collection, collection_id, credit_line, accession_year
• Status: is_public_domain, is_highlight, on_view, gallery_number
• Image: url, thumbnail_url, width, height
• AI description: alt_text, long_description, emoji_summary
• Tags: array of keywords
• Similar artworks: id, similarity_type, confidence, explanation
• Links: source_url (Met Museum page)

GET /similar/{id} - Find Similar Artworks

Find artworks similar to a given artwork using different methods.

# Precomputed (LLM-curated, recommended - includes explanations)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=precomputed&limit=10"

# Embedding-based visual similarity (CLIP)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=embedding&model=jina_clip&limit=10"

# Embedding-based conceptual similarity (text)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=embedding&model=jina_text&limit=10"

# Metadata-based (same artist, period, culture, etc.)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=metadata&limit=10"

# Combined (fuses metadata + text + image embeddings)
curl "https://museum-semantic-search.vercel.app/api/mcp/similar/met_436524?method=combined&limit=10"

Query parameters:

• method: "precomputed" (default) or "embedding"
• model: "jina_clip" (visual) or "jina_text" (conceptual) - only for embedding method
• limit: 1-50 (default 10)

Response:

{
  "meta": {
    "source_artwork": {
      "id": "met_436524",
      "title": "...",
      "artist": "..."
    },
    "method": "precomputed",
    "model": null,
    "total": 10,
    "returned": 10,
    "processing_time_ms": 45
  },
  "results": [
    {
      "id": "met_437234",
      "score": 0.92,
      "title": "...",
      "artist": "...",
      "similarity_type": "compositional",
      "similarity_explanation": "Both feature a central figure with a dog..."
    }
  ]
}

POST /image-search - Search by Image

Find visually similar artworks by uploading an image.

curl -X POST https://museum-semantic-search.vercel.app/api/mcp/image-search \
  -H "Content-Type: application/json" \
  -d '{
    "image": "data:image/jpeg;base64,/9j/4AAQ...",
    "limit": 10
  }'

Request body:

{
  "image": "base64-encoded image (with or without data URL prefix)",
  "mimeType": "image/jpeg",
  "filters": { },
  "limit": 10,
  "detail": "standard"
}

Response: Same format as /search but includes embedding_time_ms in meta.

GET /filters - Get Filter Options

Get available filter values for the collection. Supports search mode to find specific filter values.

# Get all top filter values
curl https://museum-semantic-search.vercel.app/api/mcp/filters

# Search for specific filter values (case-insensitive)
curl "https://museum-semantic-search.vercel.app/api/mcp/filters?search=japan"
curl "https://museum-semantic-search.vercel.app/api/mcp/filters?search=oil&field=mediums"

Query parameters (for search mode):

• search: Case-insensitive search string (e.g., "japan", "portrait", "oil")
• field: Limit to specific field: departments, classifications, cultures, mediums, tags, countries, periods
• limit: Max results per field (1-100, default 20)

Response (default mode):

{
  "departments": [{ "value": "European Paintings", "count": 1500 }, ...],
  "classifications": [{ "value": "Paintings", "count": 2300 }, ...],
  "cultures": [{ "value": "French", "count": 450 }, ...],
  "mediums": [{ "value": "Oil on canvas", "count": 1800 }, ...],
  "tags": [{ "value": "Portraits", "count": 800 }, ...],
  "date_range": { "min": 1250, "max": 1950 },
  "schema": {
    "artistName": { "type": "string", "description": "...", "examples": [...] },
    "medium": { "type": "string", "description": "...", "examples": [...] },
    "yearStart": { "type": "number", "description": "...", "min": 1250, "max": 1950 },
    ...
  }
}

Response (search mode):

{
  "search_query": "japan",
  "field_filter": "all",
  "results": {
    "cultures": [
      { "value": "Japan", "count": 45 },
      { "value": "probably Japan", "count": 12 }
    ]
  }
}

Workflow

Step 1: Discover the Collection

Call GET /filters first to see available departments, cultures, mediums, tags, and date ranges. Use search mode to find specific filter values.

Step 2: Search

Use POST /search - hybrid mode (default) works well for most queries.

Step 3: Get Details

Call GET /artwork/{id} on interesting results, or use detail: "full" in search to get complete details inline.

Step 4: Explore

Use GET /similar/{id} to discover related works.

Example Queries

| Query | Recommended Mode | Why | |-------|------------------|-----| | "person with a dog" | hybrid (default) | Works well for most queries | | "woman looking in mirror" | semantic | Purely conceptual description | | "stormy seascape" | hybrid | Mood-based but may match titles | | "Madonna and child" | hybrid | Common in artwork titles | | "Rembrandt self portrait" | hybrid | Artist name + description | | "paintings from 1880s" | hybrid + filters | Use yearStart/yearEnd filters |

Tips

• The collection is primarily European and American paintings from the Met Museum
• AI-generated descriptions enable finding artworks by what's depicted, not just metadata
• Use source_url to link users to the official Met Museum page
• Pagination: when has_more is true, increase offset to get more results
• For similar artworks: "precomputed" gives the best quality with explanations; "embedding" is faster
• No authentication required - the API is publicly accessible