Jhack Development Assistant

Expert guidance for using jhack utilities to streamline Juju charm development, debugging, and testing.

What is Jhack?

Jhack is a collection of developer utilities that make charm development faster and easier by:

Providing shortcuts for common operations
Enabling rapid code iteration without redeployment
Visualising charm state and relations
Capturing and replaying real events
Simplifying debugging workflows

Core Workflows

Inspecting Charm State

# View relation databags
jhack show-relation myapp postgresql
jhack show-relation myapp:database postgresql:database  # Specific endpoints

# View charm stored state
jhack show-stored myapp/0
jhack show-stored myapp/leader  # Leader unit

# List integration endpoints
jhack list-endpoints myapp

# Get charm version info
jhack charm-info myapp/0

show-relation displays what data charms are sharing in a relation as a formatted table, making debugging integration issues much easier.

show-stored shows the persistent state stored by the charm (via StoredState in Ops framework).

Live Development with Sync

# Sync local source to running unit (watch mode)
jhack sync src/ myapp/0

# Sync specific directories
jhack sync --src=./src --src=./lib myapp/0

# One-time sync (no watching)
jhack sync --no-watch src/ myapp/0

Use case: Edit code locally, jhack automatically pushes changes to the running unit. Combined with jhack utils fire, you can test changes in seconds without repacking and redeploying.

For packed charm sync: Use jhack charm sync-packed

Monitoring Events

# Watch all events in real-time
jhack tail

# Watch specific unit
jhack tail myapp/0

# Watch with debug output
jhack tail -d myapp/0

# Filter by event type
jhack tail --filter=config-changed

tail provides a colour-coded, hierarchical view of charm events as they fire, showing:

Event names
Handler execution
Nested events (e.g., relation-changed triggering config-changed)
Timing information

Firing Events Manually

# Fire a specific event
jhack utils fire myapp/0 config-changed
jhack utils fire myapp/0 database-relation-changed

# Fire with specific relation
jhack fire myapp/0 upgrade-charm

# Speed up update-status for testing
jhack ffwd myapp --interval=10s  # Every 10 seconds
jhack ffwd myapp --stop          # Reset to normal

Use case: Test specific event handlers without waiting for natural triggers or performing manual operations.

Event Replay (Advanced)

# Install recorder on unit
jhack replay install myapp/0

# List captured events
jhack replay list myapp/0

# Replay a specific event
jhack replay emit myapp/0 5  # Replay event #5

# Dump event data
jhack replay dump myapp/0 5

# Purge recorded events
jhack replay purge myapp/0

Use case: Capture real production events (with all their context and data) and replay them for debugging or testing.

Scenario Testing

# Capture current state as scenario
jhack scenario snapshot myapp/0 > state.json

# Apply a state to a unit
jhack scenario state-apply myapp/0 state.json

Integration with ops-scenario: Use snapshot to capture real charm state, then use it in unit tests with the Ops state-transition (unit) testing framework.

Charm Manipulation

# Update packed charm with local files
jhack charm update myapp.charm src/

# Sync packed charm continuously
jhack charm sync-packed --src=./src --charm=myapp.charm

# Lobotomize charm (prevent event processing)
jhack charm lobotomy myapp/0  # Enable
jhack charm lobotomy myapp/0 --undo  # Disable

# Force unit to become leader
jhack utils elect myapp/1  # Make unit 1 the leader

lobotomy is useful for freezing a charm's behavior during debugging - it prevents the charm from processing any events.

Pebble Commands (K8s Charms)

# Run pebble commands on remote units
jhack pebble -c mycontainer myapp/0 plan
jhack pebble -c mycontainer myapp/0 services
jhack pebble -c mycontainer myapp/0 logs myservice
jhack pebble -c mycontainer myapp/0 exec "ls -la /"

Shortcut for: juju ssh --container=mycontainer myapp/0 -- pebble <command>

Debugging and Logs

# Unified log view (charm + containers)
jhack debug-log myapp/0

# Execute Python in charm context
jhack script myapp/0 myscript.py

# Evaluate expression
jhack eval myapp/0 "self.unit.status"

script and eval let you run arbitrary Python code in the context of a live charm, with access to the charm instance.

Destructive Operations

# Safe removal with pattern matching
jhack nuke myapp          # Remove application
jhack nuke "test-*"       # Remove all apps matching pattern
jhack nuke --model=mymodel --all  # Nuke entire model

# Kill stuck hook execution
jhack kill myapp/0

# Auto-resolve all error states (use carefully!)
jhack utils this-is-fine

Safety: All destructive commands require confirmation unless you enable devmode: jhack conf set devmode=true

Chaos Testing

# Stress test charm through rapid changes
jhack chaos mancioppi myapp --duration=300  # 5 minutes
jhack chaos flicker myapp  # Rapid scaling up/down

Use case: Identify race conditions and state management issues by rapidly scaling, relating, and modifying charms.

Common Workflows

Rapid Development Iteration

Deploy charm: juju deploy ./myapp.charm
Start sync: jhack sync src/ myapp/0
Edit code locally
Fire event to test: jhack utils fire myapp/0 config-changed
Watch events: jhack tail myapp/0 -d
Inspect state: jhack show-stored myapp/0

Result: Test changes in seconds instead of minutes.

Debugging Integration Issues

Check relation data: jhack show-relation myapp postgresql
Fire relation event: jhack utils fire myapp/0 database-relation-changed
Watch handler execution: jhack tail myapp/0 -d
Inspect stored state: jhack show-stored myapp/0

Capturing Production Issues

Install recorder: jhack replay install myapp/0
Wait for issue to occur
List events: jhack replay list myapp/0
Dump problematic event: jhack replay dump myapp/0 <id>
Replay locally: jhack replay emit myapp/0 <id>

Testing Leadership Changes

Check current leader: juju status
Force different unit to lead: jhack utils elect myapp/1
Test leader-specific behavior
Watch events: jhack tail myapp

Command Quick Reference

# Inspection
jhack show-relation <app> <app>      # View relation data
jhack show-stored <unit>              # View stored state
jhack tail [unit]                     # Watch events
jhack charm-info <unit>               # Charm version info

# Development
jhack sync <src> <unit>               # Sync code to unit
jhack utils fire <unit> <event>       # Fire event manually
jhack ffwd <app> --interval=10s       # Speed up update-status

# Replay
jhack replay install <unit>           # Install recorder
jhack replay list <unit>              # List events
jhack replay emit <unit> <id>         # Replay event

# Scenario
jhack scenario snapshot <unit>        # Capture state
jhack scenario state-apply <unit> <file>  # Apply state

# Pebble (K8s)
jhack pebble -c <container> <unit> <command>

# Debugging
jhack debug-log <unit>                # Unified logs
jhack eval <unit> <expression>        # Evaluate in charm context
jhack script <unit> <script>          # Execute Python script

# Manipulation
jhack charm lobotomy <unit>           # Freeze charm
jhack utils elect <unit>              # Force leadership
jhack kill <unit>                     # Kill hook

# Cleanup
jhack nuke <pattern>                  # Safe removal
jhack utils this-is-fine              # Auto-resolve errors

# Chaos
jhack chaos mancioppi <app>           # Stress test

Integration with Development Tools

With Tox

# Edit charm code
vim src/charm.py

# Format and lint
tox -e format
tox -e lint

# Sync to unit
jhack sync src/ myapp/0

# Test specific handler
jhack utils fire myapp/0 config-changed

With Ops-Scenario

# Capture real state
# $ jhack scenario snapshot myapp/0 > tests/state.json

# Use in tests
import json

from ops import testing

with open('tests/state.json') as f:
    state = testing.State.from_dict(json.load(f))

# Test with real production state
ctx = testing.Context(MyCharm)
ctx.run(event, state)

With Charmcraft

# Pack charm
charmcraft pack

# Update running charm without redeploying
jhack charm update myapp.charm src/

# Or continuously sync packed charm
jhack charm sync-packed --charm=myapp.charm --src=./src

Best Practices

Development

Use sync for rapid iteration - Don't repack/redeploy for every change
Watch events with tail -d - Understand what's happening
Fire events manually - Test specific scenarios quickly
Capture real states - Use scenario snapshot for realistic test data

Debugging

Start with show-relation - Most integration issues are data problems
Check show-stored - Verify state persistence
Use replay for production issues - Capture and replay real events
Enable debug logging - jhack tail -d shows detailed output

Testing

Use scenario snapshots - Test against real charm states
Test leadership changes - Use elect to simulate failover
Stress test with chaos - Find race conditions early
Lobotomize for state inspection - Freeze charm to inspect state

Safety

Enable devmode only in dev - jhack conf set devmode=false for production
Use --dry-run - Many commands support dry-run mode
Verify nuke targets - Double-check pattern matching before confirming
Be careful with this-is-fine - Only use when you understand why units errored

Configuration

# View configuration
jhack conf show

# Set devmode (skip confirmations)
jhack conf set devmode=true

# Set log level
jhack --loglevel=DEBUG <command>

# Log to file
jhack --log-to-file=jhack.log <command>

Troubleshooting

Sync not working:

Ensure SSH access: juju ssh myapp/0
Check source path exists locally
Verify unit is active: juju status

Events not appearing in tail:

Check you're watching the right model: juju models
Switch model: juju switch <model>
Verify unit name: juju status

Replay fails:

Ensure recorder installed: jhack replay install myapp/0
Check database exists: jhack replay list myapp/0
Verify event ID: Use list output

Pebble commands fail:

Verify container name: juju ssh myapp/0 -- pebble version
Check K8s charm: Machine charms don't use Pebble
Ensure container is ready: juju status

For comprehensive troubleshooting, see references/troubleshooting.md

Resources

Jhack GitHub: https://github.com/canonical/jhack
Juju docs: https://documentation.ubuntu.com/juju/3.6/
Ops-scenario: https://documentation.ubuntu.com/ops/latest/reference/ops-testing/
Ops framework: https://documentation.ubuntu.com/ops/

Additional References

When you need detailed information:

Complete command reference: See references/command-reference.md
Troubleshooting guide: See references/troubleshooting.md
Workflow examples: See references/workflows.md

Key reminders:

Use jhack sync for rapid development iteration
tail -d is your friend for debugging
Capture production issues with replay
Test with real state using scenario snapshot
Enable devmode only in development environments