shield_lock
JinDaGe
← Back to skills
extension
Category: Development & EngineeringNo API key required

integrating-usgs-gauges

Complete USGS gauge data integration workflow from spatial discovery to model validation. Handles gauge finding, data retrieval, matching to HEC-RAS features, boundary condition generation, initial conditions, real-time monitoring, and validation metrics (NSE, KGE). Use when working with USGS data, NWIS gauges, generating boundaries from observed flow, calibrating models, validating with observed data, or setting up operational forecasting.

personAuthor: jakexiaohubgithub
open_in_newRepositorydownloadDownload package

Integrating USGS Gauges

Lightweight skill navigator - Points you to authoritative sources instead of duplicating content.

Primary Documentation Sources

COMPLETE WORKFLOWS AND API REFERENCE:

  • ras_commander/usgs/CLAUDE.md (367 lines) - Complete module documentation with workflows
    • Module overview and API reference
    • 7-stage workflow (discovery → validation)
    • Real-time monitoring patterns
    • Catalog generation workflow

WORKING EXAMPLES:

  • examples/911_usgs_gauge_data_integration.ipynb - Complete workflow (discovery → validation)
  • examples/912_usgs_real_time_monitoring.ipynb - Real-time monitoring examples
  • examples/913_bc_generation_from_live_gauge.ipynb - Boundary condition generation
  • examples/914_model_validation_with_usgs.ipynb - Model validation workflow
  • examples/910_usgs_gauge_catalog.ipynb - Catalog generation (v0.89.0+)

CODE DOCSTRINGS:

  • All classes have comprehensive docstrings with examples
  • Read docstrings for detailed parameter documentation

Quick Navigation Guide

What to Use When

| Task | Primary Source | Secondary Source | |------|----------------|------------------| | API reference | ras_commander/usgs/CLAUDE.md | Code docstrings | | Complete workflows | ras_commander/usgs/CLAUDE.md | Example notebooks | | Spatial discovery | CLAUDE.md → Spatial Queries section | 421_usgs_*.ipynb | | Data retrieval | CLAUDE.md → Data Retrieval section | 421_usgs_*.ipynb | | Gauge matching | CLAUDE.md → Gauge Matching section | 421_usgs_*.ipynb | | Boundary generation | CLAUDE.md → Boundary Conditions section | 423_bc_*.ipynb | | Initial conditions | CLAUDE.md → Initial Conditions section | 423_bc_*.ipynb | | Real-time monitoring | CLAUDE.md → Real-Time Monitoring section | 422_*.ipynb | | Catalog generation | CLAUDE.md → Catalog Generation section | 420_*.ipynb | | Model validation | CLAUDE.md → Validation Metrics section | 424_*.ipynb | | Visualization | CLAUDE.md → Visualization section | 424_*.ipynb | | Troubleshooting | Example notebooks (see errors/warnings) | Code comments |

Module Quick Reference

From ras_commander/usgs/CLAUDE.md:

Core Data Retrieval (core.py):

  • RasUsgsCore.retrieve_flow_data() - Get flow time series
  • RasUsgsCore.retrieve_stage_data() - Get stage time series
  • RasUsgsCore.get_gauge_metadata() - Get gauge info
  • RasUsgsCore.check_data_availability() - Check data exists

Spatial Discovery (spatial.py):

  • UsgsGaugeSpatial.find_gauges_in_project() - Find gauges in project bounds
  • UsgsGaugeSpatial.get_project_gauges_with_data() - Find gauges with data for sim period

Gauge Matching (gauge_matching.py):

  • GaugeMatcher.auto_match_gauges() - Automatic matching
  • GaugeMatcher.match_gauge_to_cross_section() - Match to 1D XS
  • GaugeMatcher.match_gauge_to_2d_area() - Match to 2D area

Time Series (time_series.py):

  • RasUsgsTimeSeries.resample_to_hecras_interval() - Resample to HEC-RAS intervals
  • RasUsgsTimeSeries.check_data_gaps() - Detect gaps
  • RasUsgsTimeSeries.align_timeseries() - Align observed/modeled

Boundary Generation (boundary_generation.py):

  • RasUsgsBoundaryGeneration.generate_flow_hydrograph_table() - Create BC table
  • RasUsgsBoundaryGeneration.update_boundary_hydrograph() - Update .u## file

Initial Conditions (initial_conditions.py):

  • InitialConditions.create_ic_line() - Generate IC line
  • InitialConditions.get_ic_value_from_usgs() - Extract IC from USGS data

Real-Time (real_time.py) - v0.87.0+:

  • RasUsgsRealTime.get_latest_value() - Get most recent reading
  • RasUsgsRealTime.monitor_gauge() - Continuous monitoring with callbacks
  • RasUsgsRealTime.detect_threshold_crossing() - Flood detection

Catalog (catalog.py) - v0.89.0+:

  • catalog.generate_gauge_catalog() - Create standardized gauge catalog
  • catalog.load_gauge_catalog() - Load catalog
  • catalog.load_gauge_data() - Load historical data

Validation (metrics.py):

  • metrics.nash_sutcliffe_efficiency() - NSE metric
  • metrics.kling_gupta_efficiency() - KGE metric
  • metrics.calculate_all_metrics() - Complete validation suite

Visualization (visualization.py):

  • visualization.plot_timeseries_comparison() - Observed vs modeled plots
  • visualization.plot_scatter_comparison() - Scatter with 1:1 line
  • visualization.plot_residuals() - 4-panel diagnostics

File I/O (file_io.py):

  • RasUsgsFileIo.cache_gauge_data() - Save to CSV
  • RasUsgsFileIo.load_cached_gauge_data() - Load from CSV

Minimal Quick Start

For complete quick start, see ras_commander/usgs/CLAUDE.md.

Basic pattern (copy-paste starting point):

from ras_commander import init_ras_project
from ras_commander.usgs import UsgsGaugeSpatial, RasUsgsCore, GaugeMatcher

# Initialize
init_ras_project(r"C:\Projects\MyModel", "6.5")

# 1. Find gauges
gauges = UsgsGaugeSpatial.find_gauges_in_project(
    project_folder=r"C:\Projects\MyModel",
    buffer_miles=5.0
)

# 2. Get data
flow_data = RasUsgsCore.retrieve_flow_data(
    site_no="01646500",
    start_date="2023-09-01",
    end_date="2023-09-15",
    service='iv'  # Instantaneous values
)

# 3. Match to model
matches = GaugeMatcher.auto_match_gauges(
    gauges_gdf=gauges,
    project_folder=r"C:\Projects\MyModel"
)

# For complete workflow, see ras_commander/usgs/CLAUDE.md

Common Workflow Patterns

Pattern 1: Boundary Condition Generation

See: ras_commander/usgs/CLAUDE.md → "Complete Workflow" section → Stage 5 Example: examples/913_bc_generation_from_live_gauge.ipynb

Pattern 2: Model Validation

See: ras_commander/usgs/CLAUDE.md → "Complete Workflow" section → Stage 6 Example: examples/914_model_validation_with_usgs.ipynb

Pattern 3: Real-Time Monitoring

See: ras_commander/usgs/CLAUDE.md → "Real-Time Workflows" section Example: examples/912_usgs_real_time_monitoring.ipynb

Pattern 4: Gauge Catalog Generation

See: ras_commander/usgs/CLAUDE.md → "Complete Workflow" section → Stage 7 Example: examples/910_usgs_gauge_catalog.ipynb

Troubleshooting Guide

Missing dataretrieval Module

ModuleNotFoundError: No module named 'dataretrieval'

Solution: pip install dataretrieval

No Gauges Found

Check:

  1. Increase buffer_miles parameter
  2. Verify project coordinate system
  3. Check if project bounds are correct

See: examples/911_usgs_*.ipynb for spatial discovery debugging

Data Gaps

Tools:

  • RasUsgsTimeSeries.check_data_gaps() - Detect gaps
  • RasUsgsTimeSeries.fill_data_gaps() - Interpolate gaps

See: ras_commander/usgs/CLAUDE.md → Time Series Processing

Invalid Boundary Format

Check:

  1. Verify HEC-RAS interval code (15MIN, 1HOUR, etc.)
  2. Use validate_boundary_format() to check format
  3. Review fixed-width format requirements

See: examples/913_bc_*.ipynb for boundary generation debugging

Poor Validation Metrics (NSE < 0.5)

Investigate:

  1. Model calibration parameters
  2. Boundary condition accuracy
  3. Gauge location vs model feature match
  4. Timing alignment

See: examples/914_*.ipynb for validation workflow

Dependencies

Required (always available):

  • pandas, geopandas, requests

Optional (lazy-loaded):

  • dataretrieval - USGS NWIS client (recommended)
    • Install: pip install dataretrieval
    • Methods check availability on first use

Check dependencies:

from ras_commander.usgs import check_dependencies
deps = check_dependencies()
print(deps)  # {'pandas': True, 'geopandas': True, 'dataretrieval': True/False}

Validation Metric Interpretation

See: ras_commander/usgs/CLAUDE.md → Validation Metrics section for complete details.

Quick reference:

  • NSE (Nash-Sutcliffe): −∞ to 1 (perfect = 1)

    • 0.75: Very good

    • 0.65-0.75: Good
    • 0.50-0.65: Satisfactory
    • < 0.50: Unsatisfactory
  • KGE (Kling-Gupta): −∞ to 1 (perfect = 1)
    • Similar interpretation to NSE
    • Components: correlation, bias ratio, variability ratio

For detailed interpretation: See examples/914_model_validation_with_usgs.ipynb

Key Features

Multi-Level Verifiability

  • Boundary conditions in .u## files reviewable in HEC-RAS GUI
  • Visual outputs for domain expert review
  • Code audit trails with @log_call decorators

USGS Service Compliance

  • Rate limiting (1 req/sec default)
  • Proper parameter codes (00060 = flow, 00065 = stage)
  • Timeout handling and retry logic

Data Quality

  • Automatic gap detection
  • Data availability checks
  • Resampling validation

See: ras_commander/usgs/CLAUDE.md → Key Features for details

See Also

  • HEC-RAS execution: .claude/skills/executing-hecras-plans/
  • HDF results extraction: .claude/skills/extracting-hecras-results/
  • Precipitation data: ras_commander.precipitation module
  • Boundary utilities: ras_commander.RasUnsteady

Skill Development Philosophy

This skill is intentionally lightweight (~300 lines) and serves as a navigator to authoritative sources rather than duplicating content. This approach:

  1. Eliminates duplication - Workflows exist in ras_commander/usgs/CLAUDE.md (367 lines)
  2. Reduces maintenance burden - Update workflows in one place
  3. Improves accuracy - Single source of truth for API details
  4. Provides quick navigation - Table guides you to right source for your task
  5. Keeps skills focused - Skills point to docs, docs contain complete details

When to use this skill:

  • You need quick navigation to USGS integration resources
  • You want to know which module handles which task
  • You need a starting point for USGS workflows

When to read full docs:

  • You need complete workflow details → Read ras_commander/usgs/CLAUDE.md
  • You need working code examples → Open example notebooks
  • You need parameter documentation → Read code docstrings