The Atomicity Gap: Durable Ruby Workflows with Ductwork

Outline

There is a gap in the Ruby ecosystem...

Problem

  • there are two patterns that cause silent problems with job queues
  • jobs enqueueing jobs and more than 1 side effect in a job
  • i call this the Atomicity Gap
  • Lost work or Double work

Problem transition to Solution

  • build vs. buy
  • are you really going to invest the budget for building a workflow framework and not in your product?
  • acidic_job, ActiveJob::Continuation
    • (need strong argument against here)
  • Temporal and Camunda
    • very heavyweight solutions
    • require lots of new infrastructure
  • DBOS, Hatchet, and Oban
    • do not have ruby SDKs

Solution

  • durable workflows with Ductwork
  • exactly-once step transitions guarantee
  • at-least-once step executions made safe with idempotency keys
  • reaper cleans up after a crash

Closing

  • call-to-action: install LLM agent to evaluate and report where your gaps live in your codebase