Bio Reads QC Mapping

Ingest, QC, and map reads with reproducible outputs. Use for raw read processing and coverage stats.

Instructions

Tool guides and versions: docs/README.md.

Parse and validate sample_sheet.tsv against schemas/sample-sheet.schema.json. Use the executable driver for both planning and restartable execution:
```
uv run --script skills/bio-reads-qc-mapping/scripts/run_reads_qc_mapping.py \
  sample_sheet.tsv --out results/bio-reads-qc-mapping
# Inspect run_manifest.json, then execute the same plan:
uv run --script skills/bio-reads-qc-mapping/scripts/run_reads_qc_mapping.py \
  sample_sheet.tsv --out results/bio-reads-qc-mapping --execute
```
read_type must be paired_short, single_short, or long. Mapping runs only for rows with a non-empty reference; a missing reference is not a mapping failure. The driver reuses a stage only when its declared outputs are non-empty and the stage's .done marker exists.
For short reads: run QC and adapter/quality trimming with bbduk or fastp v1.3.3+.
For long reads: use current basecaller-aware QC first. For ONT, prefer Dorado summaries/trimming during basecalling or demultiplexing when starting from signal/BAM; for FASTQ-only filtering use chopper for quality/length/end trimming or filtlong v0.3.1 when selecting reads for assembly (v0.3.0 renamed the short-read options to --short_1 / --short_2; see docs/filtlong.md). Use Pychopper for full-length cDNA. Treat Porechop_ABI as a targeted legacy/fallback adapter-discovery tool, and record why it is needed.
- For very large ONT FASTQ inputs, do not burn the first full read pass on raw gzip -t or raw seqkit stats preflight unless the user explicitly asks for it. Record raw stat metadata and, if needed, a small sampled sanity check; let the first full pass be the actual filtering/orientation step, then run seqkit stats on produced outputs.
- For ONT cDNA with Pychopper, write outputs with plain .fastq suffixes unless you explicitly pipe/compress them yourself. Pychopper can write plain FASTQ even when the output path ends in .gz; avoid gzip -t on Pychopper outputs unless magic bytes confirm gzip. If legacy outputs have .fastq.gz names but plain FASTQ content, rename them to .fastq before resuming.
- Pychopper report plotting can fail after the reads are already processed, for example from a pandas/statistics type-conversion error. On that failure, inspect whether the classified/unclassified/rescued/read-stats outputs exist and are non-empty. If they do, resume downstream from those outputs rather than rerunning the full Pychopper pass.
Map reads and produce coverage tables:
- Short reads, CPU: bbmap or bwa-mem2 v2.2.1+. Short reads, GPU node available: NVIDIA Parabricks fq2bam (wraps bwa-mem2 + GATK markdup; typically 3–4× faster than bwa-mem2 on 8 cores and up to ~80× over a 96-core CPU pipeline).
- Long reads, CPU: minimap2 v2.30+. AVX-512 hardware: mm2-fast as a drop-in replacement (~1.8× speedup). GPU node available: mm2-gb or mm2-ax for CUDA-accelerated long-read alignment.
Record the tool, version, and any GPU device used in the run log.

Input Requirements

Prerequisites:

Tools declared in the project's pinned Pixi environment. See docs/README.md for expected tools.
Sample sheet and reads are available. Inputs:
sample_sheet.tsv
reads/*.fastq.gz
reference.fasta (optional)

Output

results/bio-reads-qc-mapping/trimmed_reads/
results/bio-reads-qc-mapping/qc_reports/
results/bio-reads-qc-mapping/mapping_stats.tsv (mapping_status is planned before --execute, then completed or reused, and not_requested for rows with no reference)
results/bio-reads-qc-mapping/logs/
stdout: the last line is one JSON envelope {ok, skill, out, manifest, warnings} (driver stdout contract in AGENTS.md)
Sample sheet contract: schemas/sample-sheet.schema.json

Quality Gates

[ ] Post-QC read count sanity checks pass.
[ ] Mapping rate meets project thresholds.
[ ] On execution failure, preserve logs and report the failed command; retry only after diagnosing the cause and recording the changed parameters. Report unmet biological thresholds as results; never tune parameters solely to pass a gate.
[ ] Validate sample sheet schema and FASTQ integrity.
[ ] The plan covers every sheet row exactly once, and mapping gates are applied only to rows that supplied a reference.
[ ] For long-read QC, record whether trimming happened in the basecaller/demultiplexer, chopper, filtlong, Pychopper, or a documented Porechop_ABI fallback.
[ ] For huge ONT inputs, avoid redundant full-file raw preflights; document raw file size/mtime and make the first full pass productive.
[ ] For Pychopper outputs, verify actual file type by content, not suffix. Plain FASTQ with a .gz suffix must be renamed or explicitly compressed before downstream tools that expect gzip.
[ ] Resume guards skip a stage only when its expected outputs are non-empty and its .done marker exists. Run a lightweight content check (seqkit stats, FASTQ header sniff, or gzip magic as appropriate) before accepting downstream data.

Examples

The runnable fixture at fixtures/sample_sheet.tsv covers paired-end, single-end, and long reads.

Troubleshooting

Issue: Pychopper failed during report/stat plotting but output FASTQs exist Solution: Treat this as a recoverable post-processing failure. Confirm the classified FASTQ is non-empty and readable, fix any misleading .gz suffix, run seqkit stats, and resume downstream steps from the existing Pychopper outputs.