Celery Task Queue Architect
Expert guidance for designing and operating Celery-based distributed task queues — from broker selection and queue topology to retry strategies, idempotency patterns, worker scaling, and production monitoring.
SupaScore
83.3Best for
- ▸Design scalable Celery task queue architecture for high-volume applications processing millions of tasks daily
- ▸Debug and optimize existing Celery deployments experiencing message loss, worker bottlenecks, or retry failures
- ▸Implement robust task retry strategies with exponential backoff and dead letter queues for production systems
- ▸Configure RabbitMQ or Redis message brokers with proper durability, routing, and high availability for Celery
- ▸Design idempotent task patterns and queue topology for complex workflows with task chaining and fan-out patterns
What you'll get
- ●Complete Celery configuration with broker-specific settings, retry decorators, and monitoring setup for a specific workload pattern
- ●Queue topology design with routing keys, exchange configuration, and worker deployment strategy for multi-environment setup
- ●Production-ready task implementation with idempotency patterns, error handling, and performance optimization techniques
Not designed for ↓
- ×General Python async/await programming or basic task scheduling without distributed queue requirements
- ×Real-time communication systems like WebSockets or chat applications requiring immediate bidirectional messaging
- ×Simple cron-based job scheduling that doesn't require distributed processing or complex retry logic
Specific details about task volume, latency requirements, failure scenarios, current architecture constraints, and the types of workloads (CPU-bound, I/O-bound, scheduled batch jobs) being processed.
Detailed architectural recommendations with specific Celery configuration code, broker setup instructions, queue topology diagrams, and production-ready monitoring and retry strategies.
Evidence Policy
Enabled: this skill cites sources and distinguishes evidence from opinion.
Research Foundation: 7 sources (3 official docs, 2 books, 1 web, 1 industry frameworks)
This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.
Version History
Initial release
Prerequisites
Use these skills first for best results.
Works well with
Need more depth?
Specialist skills that go deeper in areas this skill touches.
Common Workflows
Distributed System Architecture Design
Complete workflow for designing, implementing, monitoring, and operating a distributed task processing system from API endpoints through background processing to production reliability.
Activate this skill in Claude Code
Sign up for free to access the full system prompt via REST API or MCP.
Start Free to Activate This Skill© 2026 Kill The Dragon GmbH. This skill and its system prompt are protected by copyright. Unauthorised redistribution is prohibited. Terms of Service · Legal Notice