Video Upload Processing

Skill Profile

(Select at least one profile to enable specific modules)

• [ ] DevOps
• [x] Backend
• [ ] Frontend
• [ ] AI-RAG
• [ ] Security Critical

Overview

Video upload and processing handles file uploads, transcoding, thumbnail generation and delivery. This guide covers S3 storage, FFmpeg processing, and CDN delivery for building video processing systems that handle uploads efficiently and deliver optimized content.

Why This Matters

Video upload and processing is critical for video platforms as it directly impacts:

• User Experience: Fast uploads and quick availability
• Cost: Optimized storage and delivery costs
• Quality: Consistent transcoding across all content
• Scalability: Handle concurrent uploads efficiently

---

Core Concepts

1. Direct Upload: Client uploads directly to S3
2. Resumable Upload: Support for large files
3. Multipart Upload: Split large files into chunks
4. Processing Queue: Async processing with job queues
5. Thumbnail Generation: Create preview images
6. Metadata Extraction: Extract video information
7. Progress Tracking: Real-time progress updates

Inputs / Outputs / Contracts

Skill Composition

• Depends on: None
• Compatible with: None
• Conflicts with: None
• Related Skills: None

Quick Start / Implementation Example

1. Review requirements and constraints
2. Set up development environment
3. Implement core functionality following patterns
4. Write tests for critical paths
5. Run tests and fix issues
6. Document any deviations or decisions

# Example implementation following best practices
def example_function():
    # Your implementation here
    pass

Assumptions

• Sufficient S3 storage capacity
• FFmpeg installed and accessible
• Network connectivity for S3 uploads
• Queue system available (Redis/BullMQ)

Compatibility

| Upload Method | Max Size | Resumable | Browser Support | |---------------|-----------|------------|-----------------| | Direct Upload | 5GB | No | All | | Resumable | 5GB | Yes | Modern browsers | | Multipart | 5TB | Yes | All |

---

Test Scenario Matrix (QA Strategy)

| Type | Focus Area | Required Scenarios / Mocks | | :--- | :--- | :--- | | Unit | Core Logic | Must cover primary logic and at least 3 edge/error cases. Target minimum 80% coverage | | Integration | DB / API | All external API calls or database connections must be mocked during unit tests | | E2E | User Journey | Critical user flows to test | | Performance | Latency / Load | Benchmark requirements | | Security | Vuln / Auth | SAST/DAST or dependency audit | | Frontend | UX / A11y | Accessibility checklist (WCAG), Performance Budget (Lighthouse score) |

Technical Guardrails & Security Threat Model

1. Security & Privacy (Threat Model)

• Top Threats: Injection attacks, authentication bypass, data exposure

• [ ] Data Handling: Sanitize all user inputs to prevent Injection attacks. Never log raw PII
• [ ] Secrets Management: No hardcoded API keys. Use Env Vars/Secrets Manager
• [ ] Authorization: Validate user permissions before state changes

2. Performance & Resources

• [ ] Execution Efficiency: Consider time complexity for algorithms
• [ ] Memory Management: Use streams/pagination for large data
• [ ] Resource Cleanup: Close DB connections/file handlers in finally blocks

3. Architecture & Scalability

• [ ] Design Pattern: Follow SOLID principles, use Dependency Injection
• [ ] Modularity: Decouple logic from UI/Frameworks

4. Observability & Reliability

• [ ] Logging Standards: Structured JSON, include trace IDs request_id
• [ ] Metrics: Track error_rate, latency, queue_depth
• [ ] Error Handling: Standardized error codes, no bare except
• [ ] Observability Artifacts:
  • Log Fields: timestamp, level, message, request_id
  • Metrics: request_count, error_count, response_time
  • Dashboards/Alerts: High Error Rate > 5%

Agent Directives

1. Always use resumable uploads for large files
2. Queue processing jobs for async handling
3. Track progress and notify users
4. Generate multiple thumbnails at different timestamps
5. Extract metadata for search and display
6. Clean up temporary files after processing
7. Use CDN for content delivery

Definition of Done (DoD) Checklist

• [ ] Tests passed + coverage met
• [ ] Lint/Typecheck passed
• [ ] Logging/Metrics/Trace implemented
• [ ] Security checks passed
• [ ] Documentation/Changelog updated
• [ ] Accessibility/Performance requirements met (if frontend)

Anti-patterns / Pitfalls

• ⛔ Don't: Log PII, catch-all exception, N+1 queries
• ⚠️ Watch out for: Common symptoms and quick fixes
• 💡 Instead: Use proper error handling, pagination, and logging

Reference Links & Examples

• Internal documentation and examples
• Official documentation and best practices
• Community resources and discussions

Versioning & Changelog

• Version: 1.0.0
• Changelog:
  • 2026-02-22: Initial version with complete template structure