Distributed Workflow Engine
Fault-tolerant job scheduling with DAG orchestration
A production-grade distributed job scheduler with priority queues, concurrent worker pools, at-least-once delivery, and DAG workflow orchestration — built from scratch in Go.
Modern applications need reliable background job processing that handles failures gracefully. Off-the-shelf solutions often lack fine-grained control over scheduling semantics, retry policies, and dependency-aware execution.
Built a custom job scheduler in Go with Redis for priority dispatch (using atomic Lua scripts with 6:3:1 weighted scheduling) and PostgreSQL for durable state transitions. DAG workflows use Kahn's algorithm for cycle detection and dependency-aware parallel execution.