Sensor Integration Skill

Version: 1.0.0 | Alignment: Constitution v6.0.0, Pyodide-Compatible Code | Purpose: Generate educational sensor integration code for browser-based learning

Purpose

Generate production-quality Python code for sensor integration that:

Runs in Pyodide (browser environment)
Demonstrates sensor data processing and fusion
Simulates sensor noise and calibration
Provides educational value for perception systems
Supports InteractivePython components in Docusaurus

Key Focus: Teaching how robots perceive the world through sensors, not just using sensor libraries.

When to Activate

Activate when:

Creating code examples for IMU sensors (accelerometer, gyroscope, magnetometer)
Demonstrating sensor fusion algorithms (Kalman filtering, complementary filters)
Teaching depth perception (stereo vision, LiDAR simulation)
Creating tactile sensor examples
Building multi-sensor perception systems

Trigger phrases:

"Create sensor fusion code"
"Generate IMU sensor example"
"Build Kalman filter for sensors"
"Create depth perception simulation"
"Generate sensor calibration code"

Technical Capabilities

Supported Sensor Types

IMU Sensors
- Accelerometer (3-axis acceleration)
- Gyroscope (angular velocity)
- Magnetometer (magnetic field)
- Sensor fusion (attitude estimation)
Depth Sensors
- Simulated LiDAR (2D/3D point clouds)
- Depth camera simulation
- Stereo vision depth estimation
Tactile Sensors
- Force/torque sensors
- Contact detection
- Pressure distribution
Sensor Fusion
- Kalman filtering
- Complementary filters
- Multi-sensor state estimation

Pyodide-Compatible Libraries

✅ Supported:

NumPy (arrays, linear algebra, signal processing)
Matplotlib (plotting sensor data, visualizations)
SciPy (filtering, optimization, signal processing)
Math (basic math functions)

❌ NOT Supported:

ROS (Robot Operating System)
Sensor hardware drivers
Real-time sensor libraries
File I/O (reading sensor logs)

Code Generation Standards

Code Structure

"""
[Sensor type] Integration Example

Educational Purpose: [What students will learn about sensors]
Physical AI Context: [How real robots use this sensor]
"""

import numpy as np
import matplotlib.pyplot as plt
from typing import Tuple, List, Optional

# Type hints for all functions
# Comprehensive docstrings (Google style)
# Error handling (try-except where appropriate)
# Noise modeling for realistic sensor behavior
# Calibration examples

Example Template: IMU Sensor Fusion

"""
IMU Sensor Fusion using Complementary Filter

Educational Purpose: Students learn how to combine accelerometer and gyroscope data
Physical AI Context: Real robots use sensor fusion for accurate attitude estimation
"""

import numpy as np
import matplotlib.pyplot as plt
from typing import Tuple, List

class IMUSensor:
    """Simulated IMU sensor with noise."""
    
    def __init__(self, noise_level: float = 0.1):
        """
        Initialize IMU sensor.
        
        Args:
            noise_level: Standard deviation of sensor noise
        """
        self.noise_level = noise_level
    
    def read_accelerometer(self, true_accel: np.ndarray) -> np.ndarray:
        """
        Read accelerometer with noise.
        
        Args:
            true_accel: True acceleration [ax, ay, az]
        
        Returns:
            Noisy acceleration measurement
        """
        noise = np.random.normal(0, self.noise_level, 3)
        return true_accel + noise
    
    def read_gyroscope(self, true_angular_vel: np.ndarray) -> np.ndarray:
        """
        Read gyroscope with noise.
        
        Args:
            true_angular_vel: True angular velocity [wx, wy, wz]
        
        Returns:
            Noisy angular velocity measurement
        """
        noise = np.random.normal(0, self.noise_level, 3)
        return true_angular_vel + noise

def complementary_filter(
    accel_data: np.ndarray,
    gyro_data: np.ndarray,
    dt: float,
    alpha: float = 0.98
) -> float:
    """
    Complementary filter for attitude estimation.
    
    Combines accelerometer (low-frequency) and gyroscope (high-frequency) data.
    
    Args:
        accel_data: Accelerometer reading [ax, ay, az]
        gyro_data: Gyroscope reading [wx, wy, wz]
        dt: Time step (seconds)
        alpha: Filter coefficient (0-1), higher = trust gyro more
    
    Returns:
        Estimated pitch angle (radians)
    """
    # Calculate pitch from accelerometer
    pitch_accel = np.arctan2(accel_data[0], np.sqrt(accel_data[1]**2 + accel_data[2]**2))
    
    # Integrate gyroscope
    pitch_gyro = pitch_gyro + gyro_data[1] * dt
    
    # Complementary filter: blend accel (low-freq) and gyro (high-freq)
    pitch = alpha * pitch_gyro + (1 - alpha) * pitch_accel
    
    return pitch

def simulate_imu_fusion(
    duration: float = 10.0,
    dt: float = 0.01,
    noise_level: float = 0.1
):
    """
    Simulate IMU sensor fusion.
    
    Args:
        duration: Simulation duration (seconds)
        dt: Time step (seconds)
        noise_level: Sensor noise level
    """
    imu = IMUSensor(noise_level=noise_level)
    
    time = np.arange(0, duration, dt)
    true_pitch = 0.5 * np.sin(0.5 * time)  # True pitch motion
    estimated_pitch = []
    
    pitch_gyro = 0.0  # Initialize
    
    for t in time:
        # True sensor values (simplified)
        true_accel = np.array([np.sin(true_pitch[int(t/dt)]), 0, np.cos(true_pitch[int(t/dt)])])
        true_gyro = np.array([0, 0.5 * 0.5 * np.cos(0.5 * t), 0])  # Derivative of pitch
        
        # Read sensors
        accel = imu.read_accelerometer(true_accel)
        gyro = imu.read_gyroscope(true_gyro)
        
        # Sensor fusion
        pitch = complementary_filter(accel, gyro, dt)
        estimated_pitch.append(pitch)
        pitch_gyro = pitch
    
    # Visualize
    fig, ax = plt.subplots(figsize=(10, 6))
    ax.plot(time, np.degrees(true_pitch), 'b-', label='True Pitch', linewidth=2)
    ax.plot(time, np.degrees(estimated_pitch), 'r--', label='Estimated Pitch', linewidth=2)
    ax.set_xlabel('Time (seconds)')
    ax.set_ylabel('Pitch Angle (degrees)')
    ax.set_title('IMU Sensor Fusion: Complementary Filter')
    ax.legend()
    ax.grid(True, alpha=0.3)
    plt.tight_layout()
    plt.show()

# Example usage
if __name__ == "__main__":
    simulate_imu_fusion()

Example Template: LiDAR Simulation

"""
2D LiDAR Simulation

Educational Purpose: Students learn how LiDAR sensors work and process point clouds
Physical AI Context: Real robots use LiDAR for mapping and obstacle detection
"""

import numpy as np
import matplotlib.pyplot as plt
from typing import List, Tuple

def simulate_lidar_2d(
    robot_pos: Tuple[float, float],
    robot_angle: float,
    obstacles: List[Tuple[float, float, float]],  # (x, y, radius)
    max_range: float = 5.0,
    num_beams: int = 360
) -> Tuple[np.ndarray, np.ndarray]:
    """
    Simulate 2D LiDAR scan.
    
    Args:
        robot_pos: Robot position (x, y)
        robot_angle: Robot orientation (radians)
        obstacles: List of circular obstacles (x, y, radius)
        max_range: Maximum LiDAR range (meters)
        num_beams: Number of LiDAR beams
    
    Returns:
        (angles, ranges) - LiDAR scan data
    """
    angles = np.linspace(0, 2*np.pi, num_beams)
    ranges = np.full(num_beams, max_range)
    
    for i, angle in enumerate(angles):
        beam_angle = robot_angle + angle
        
        # Ray casting
        for obstacle in obstacles:
            # Calculate intersection with circular obstacle
            # (simplified ray-circle intersection)
            dx = np.cos(beam_angle)
            dy = np.sin(beam_angle)
            
            # Ray from robot
            ox, oy = robot_pos
            cx, cy, r = obstacle
            
            # Vector from robot to obstacle center
            to_center = np.array([cx - ox, cy - oy])
            to_center_dist = np.linalg.norm(to_center)
            
            # Project onto ray
            proj = np.dot(to_center, [dx, dy])
            
            if proj > 0:  # Obstacle in front
                perp_dist = np.sqrt(to_center_dist**2 - proj**2)
                if perp_dist < r:  # Ray intersects obstacle
                    # Calculate intersection distance
                    intersection_dist = proj - np.sqrt(r**2 - perp_dist**2)
                    if 0 < intersection_dist < ranges[i]:
                        ranges[i] = intersection_dist
    
    return angles, ranges

def visualize_lidar_scan(
    robot_pos: Tuple[float, float] = (0, 0),
    robot_angle: float = 0,
    obstacles: List[Tuple[float, float, float]] = [(2, 1, 0.5), (-1, 2, 0.3)]
):
    """Visualize 2D LiDAR scan."""
    angles, ranges = simulate_lidar_2d(robot_pos, robot_angle, obstacles)
    
    # Convert to Cartesian
    x_points = robot_pos[0] + ranges * np.cos(robot_angle + angles)
    y_points = robot_pos[1] + ranges * np.sin(robot_angle + angles)
    
    fig, ax = plt.subplots(figsize=(10, 10))
    
    # Draw obstacles
    for ox, oy, r in obstacles:
        circle = plt.Circle((ox, oy), r, color='gray', alpha=0.5)
        ax.add_patch(circle)
    
    # Draw LiDAR scan
    ax.scatter(x_points, y_points, c=ranges, cmap='viridis', s=10)
    
    # Draw robot
    ax.plot(robot_pos[0], robot_pos[1], 'ro', markersize=15, label='Robot')
    ax.arrow(robot_pos[0], robot_pos[1], 
             0.5*np.cos(robot_angle), 0.5*np.sin(robot_angle),
             head_width=0.1, head_length=0.1, fc='red', ec='red')
    
    ax.set_xlim(-3, 3)
    ax.set_ylim(-3, 3)
    ax.set_aspect('equal')
    ax.grid(True, alpha=0.3)
    ax.set_xlabel('X (meters)')
    ax.set_ylabel('Y (meters)')
    ax.set_title('2D LiDAR Scan Simulation')
    ax.legend()
    
    plt.tight_layout()
    plt.show()

Best Practices

1. Realistic Noise Modeling

Add Gaussian noise to sensor readings
Model sensor drift (gyroscope bias)
Include quantization effects
Show how noise affects fusion algorithms

2. Calibration Examples

Sensor offset calibration
Scale factor calibration
Coordinate frame alignment
Temperature compensation (conceptual)

3. Educational Clarity

Show raw sensor data vs. processed data
Visualize sensor fusion process step-by-step
Compare different fusion algorithms
Demonstrate sensor limitations

4. Code Quality

Type hints for all functions
Comprehensive docstrings
Error handling for invalid inputs
Clear variable names

Common Patterns

Pattern 1: Sensor Reading with Noise

def read_sensor(true_value, noise_std):
    noise = np.random.normal(0, noise_std)
    return true_value + noise

Pattern 2: Kalman Filter

def kalman_filter(measurement, state_estimate, uncertainty):
    # Prediction step
    # Update step
    # Return new estimate
    pass

Pattern 3: Sensor Calibration

def calibrate_sensor(raw_readings, known_values):
    # Calculate offset and scale
    # Return calibration parameters
    pass

Pattern 4: Multi-Sensor Fusion

def fuse_sensors(imu_data, camera_data, lidar_data):
    # Combine data from multiple sensors
    # Weight by sensor reliability
    # Return fused estimate
    pass

Integration with InteractivePython

Usage in MDX Files

<InteractivePython>
{`
import numpy as np
import matplotlib.pyplot as plt

# [Generated sensor code here]
`}
</InteractivePython>

Student Modifications

Encourage students to:

Adjust noise levels and observe effects
Modify filter parameters
Change sensor configurations
Experiment with different fusion algorithms

Troubleshooting

Issue: Sensor fusion unstable

Solution: Check filter parameters, ensure proper initialization, validate sensor data ranges.

Issue: Visualization unclear

Solution: Add labels, legends, use appropriate color maps, show multiple views.

Issue: Code too slow

Solution: Use vectorized NumPy operations, reduce number of sensor readings, optimize algorithms.

Output Format

When generating code:

Header Comment: Clear description and educational purpose
Sensor Class/Function: Simulated sensor with noise
Processing Functions: Fusion, filtering, calibration
Visualization: Clear plots showing sensor data and processing results
Example Usage: Working example demonstrating functionality

Remember: You're teaching how robots perceive the world. Focus on understanding sensor principles, noise, and fusion algorithms, not just using sensor libraries.