Continuous Skill Polishing

Purpose

Convert repeated tasks into skills, version control them in Git, and polish iteratively based on usage feedback to achieve 20% consistency gain.

When to Use

Identifying repeated task patterns
Improving skill consistency
Addressing skill failure cases
Tracking skill evolution over time
Refining based on user feedback
Building team-specific expertise

Core Instructions

Step 1: Identify Repeated Task

def identify_repeated_tasks(task_history):
    """
    Analyze task history to find repeated patterns
    """
    # Cluster similar tasks
    clusters = cluster_tasks_by_similarity(task_history)

    # Find clusters with multiple instances
    repeated = [
        cluster for cluster in clusters
        if len(cluster) >= 3  # Repeated 3+ times
    ]

    return repeated


def should_create_skill(task_cluster):
    """
    Determine if cluster warrants a skill
    """
    criteria = {
        'frequency': len(task_cluster) >= 3,  # Repeated often
        'consistency': has_consistent_pattern(task_cluster),  # Similar approach
        'complexity': is_sufficiently_complex(task_cluster),  # Not trivial
        'value': provides_reusable_value(task_cluster)  # Worth capturing
    }

    return all(criteria.values())

Step 2: Create Initial Skill

# Identify repeated task
task="Transform CSV to JSON with validation"

# Create skill directory
mkdir -p .claude/skills/csv-to-json-transformer

# Generate initial SKILL.md
cat > .claude/skills/csv-to-json-transformer/SKILL.md << 'EOF'
---
name: csv-to-json-transformer
description: Transform CSV files to JSON with schema validation...
---

# CSV to JSON Transformer

[Initial implementation based on repeated pattern]
EOF

# Version control
git add .claude/skills/csv-to-json-transformer
git commit -m "feat: add csv-to-json-transformer skill (v1.0.0)"

Step 3: Track Usage and Failures

class SkillTracker:
    """Track skill usage and performance"""

    def __init__(self, skill_name):
        self.skill_name = skill_name
        self.usage_log = []

    def log_usage(self, task, result):
        """Log each skill usage"""
        self.usage_log.append({
            'timestamp': datetime.now(),
            'task': task,
            'success': result.success,
            'issues': result.issues,
            'feedback': result.user_feedback
        })

    def analyze_failures(self):
        """Identify patterns in failures"""
        failures = [
            log for log in self.usage_log
            if not log['success']
        ]

        # Group by issue type
        issues_by_type = {}
        for failure in failures:
            for issue in failure['issues']:
                issue_type = classify_issue(issue)
                if issue_type not in issues_by_type:
                    issues_by_type[issue_type] = []
                issues_by_type[issue_type].append(failure)

        return issues_by_type

    def calculate_consistency(self):
        """Calculate skill consistency score"""
        if not self.usage_log:
            return 0.0

        successes = sum(1 for log in self.usage_log if log['success'])
        return successes / len(self.usage_log)

Step 4: Polish Based on Feedback

# Analyze failure patterns
python analyze_skill.py csv-to-json-transformer

# Output:
# Issue type: encoding_errors (3 occurrences)
# Issue type: missing_fields (5 occurrences)
# Issue type: malformed_csv (2 occurrences)

# Update skill to address issues
cat >> .claude/skills/csv-to-json-transformer/SKILL.md << 'EOF'

## Error Handling

### Encoding Issues
Handle various encodings (UTF-8, Latin-1, etc.):
```python
encodings = ['utf-8', 'latin-1', 'cp1252']
for encoding in encodings:
    try:
        df = pd.read_csv(file, encoding=encoding)
        break
    except UnicodeDecodeError:
        continue

Missing Fields

Validate against schema and provide defaults:

required_fields = ['id', 'name', 'email']
for field in required_fields:
    if field not in df.columns:
        df[field] = None  # or default value

EOF

Version control the improvement

git add .claude/skills/csv-to-json-transformer/SKILL.md git commit -m "polish: improve error handling for encoding and missing fields" git tag csv-to-json-v1.1.0


### Step 5: Measure Improvement

```python
def measure_polish_impact(skill_name, before_version, after_version):
    """
    Compare skill performance before and after polishing
    """
    before_logs = load_usage_logs(skill_name, before_version)
    after_logs = load_usage_logs(skill_name, after_version)

    before_consistency = calculate_consistency(before_logs)
    after_consistency = calculate_consistency(after_logs)

    improvement = ((after_consistency - before_consistency) /
                   before_consistency * 100)

    return {
        'before': before_consistency,
        'after': after_consistency,
        'improvement_percent': improvement
    }

# Example output:
# {
#     'before': 0.75,  # 75% success rate
#     'after': 0.90,   # 90% success rate
#     'improvement_percent': 20.0  # 20% improvement
# }

Polish Workflow

1. CREATE
   └─> Initial skill from repeated pattern
       └─> Git commit: "feat: add skill (v1.0.0)"

2. USE
   └─> Apply skill to tasks
       └─> Track successes and failures

3. ANALYZE
   └─> Identify failure patterns
       └─> Group by issue type
       └─> Prioritize most common issues

4. POLISH
   └─> Update skill to address issues
       └─> Add error handling
       └─> Improve instructions
       └─> Add examples
       └─> Git commit: "polish: [specific improvement]"

5. MEASURE
   └─> Compare before/after consistency
       └─> If improved: keep changes
       └─> If worse: revert and try different approach

6. REPEAT
   └─> Continue polishing based on new feedback

Versioning Strategy

# Semantic versioning for skills
# v[MAJOR].[MINOR].[PATCH]

# MAJOR: Breaking changes to skill interface
git commit -m "feat!: change required fields (BREAKING CHANGE)"
git tag csv-to-json-v2.0.0

# MINOR: New features, backwards compatible
git commit -m "feat: add Excel support"
git tag csv-to-json-v1.2.0

# PATCH: Bug fixes, polishing
git commit -m "fix: handle empty CSV files"
git tag csv-to-json-v1.1.1

Example: Skill Evolution

v1.0.0 - Initial

---
name: csv-to-json-transformer
description: Transform CSV to JSON
---
# Basic CSV to JSON transformation

Consistency: 75%

v1.1.0 - Error Handling Polish

---
name: csv-to-json-transformer
description: Transform CSV to JSON with encoding detection and validation
---
# CSV to JSON with error handling
- Handle multiple encodings
- Validate required fields

Consistency: 85% (+10%)

v1.2.0 - Feature Addition

---
name: csv-to-json-transformer
description: Transform CSV/Excel to JSON with schema validation and data cleaning
---
# CSV/Excel to JSON
- Multiple format support
- Schema validation
- Data cleaning and normalization

Consistency: 90% (+5%)

Total improvement: v1.0.0 → v1.2.0 = 20% consistency gain

Best Practices

When to Create a Skill

Task repeated 3+ times
Consistent approach across instances
Sufficiently complex (not trivial)
Provides reusable value

When to Polish

Failure rate > 10%
User feedback indicates confusion
New edge cases discovered
Competing better approaches emerge

Polishing Priorities

Fix high-frequency failures first
Improve unclear instructions
Add missing error handling
Provide better examples
Optimize performance

Git Workflow

Use feature branches for major changes
Tag stable versions
Keep CHANGELOG.md updated
Document breaking changes

Performance Characteristics

20% consistency improvement through iterative polishing
Git version history provides complete evolution trail
Measurable progress via before/after metrics
Team collaboration through shared skill repository

Version

v1.0.0 (2025-10-23) - Based on skill evolution patterns