Walking through the architecture decisions, trade-offs, and Python/FFmpeg pipeline that powers Full Battery Media's content distribution across 6+ platforms simultaneously.
The hardest constraint was latency. Content creators needed uploads to be published across YouTube, TikTok, Instagram, and three other platforms within 15 minutes of upload — including 4K transcoding.
Our solution was a multi-stage async pipeline: raw upload → S3 → SQS trigger → FFmpeg workers (containerised on ECS) → parallel publish workers per platform. The key insight was isolating the transcoding from the publishing — two different scaling characteristics.
The transcoding layer is CPU-bound and predictable. We use spot instances on ECS and over-provision by 30% during business hours. Each worker handles one file at a time but we run many workers in parallel — horizontal scaling is trivial because transcoding is stateless.
The publishing layer is I/O-bound and unpredictable — each platform API has different rate limits, auth flows, and reliability characteristics. We model each platform as a separate queue consumer with its own retry logic, backoff strategy, and dead letter queue.
The CMS itself is a simple React + TypeScript frontend backed by a FastAPI service. Content creators upload once; the system handles the rest. Status updates flow back via Server-Sent Events so creators see real-time progress on each platform.
The biggest lesson: don't build a monolithic pipeline. Each stage of the process has fundamentally different scaling and failure characteristics. Embrace the seams.