Core flows

Outbox & notifications

When a booking is confirmed, the driver gets an email, a push notification and an in-app message, even if the email provider is down at that moment. The transactional outbox makes that reliable without slowing the request down.

The dual-write problem#

Saving a booking and sending an email are two systems. Send first and the save may fail: an email for a booking that does not exist. Save first and the process may crash before sending: a booking nobody hears about. The outbox removes the second system from the request entirely.

BEGIN

one Postgres transaction

Inside the request that changes state.

Update the booking

status → confirmed

INSERT outbox_events

type booking.confirmed, payload, available_at = now

COMMIT

Both rows exist, or neither does. The API answers the driver immediately.

Worker delivers

Celery

Woken right after commit; a sweep every 30 s is the safety net if the wake-up is lost.

Rendered template via Resend

Push

Expo push to the user's devices

In-app

Row in the notifications inbox

The event row is written in the same transaction as the state change, so it exists if and only if the change committed. Delivery happens afterwards, in a worker.

Many workers, no double delivery#

Workers claim events with a lease. Rows are locked with SKIP LOCKED, so concurrent workers grab different rows instead of waiting on each other, and available_at is pushed into the future before the lock is released. If a worker crashes mid-delivery the lease simply runs out and another worker retries the event. Delivery is at-least-once, so every handler is idempotent.

app/workers/outbox_processor.py (simplified)python

rows = (await session.execute(
    select(OutboxEvent)
    .where(
        OutboxEvent.processed_at.is_(None),
        OutboxEvent.available_at <= now,
        OutboxEvent.attempts < self._max_attempts,   # 10, then it is a dead letter
    )
    .order_by(OutboxEvent.available_at)
    .limit(self._batch)                      # 50 per batch
    .with_for_update(skip_locked=True)       # other workers skip these rows
)).scalars()
for row in rows:
    row.attempts += 1
    row.available_at = now + backoff(row.attempts, self._base, self._max)   # the lease
await session.commit()

Retries with exponential backoff and jitter#

A failed delivery is retried after 15 s, 30 s, 1 min, 2 min … capped at one hour, for up to 10 attempts. Each delay is randomised to 75–100% of its value, so thousands of events that failed together during a provider outage do not all retry in the same second and trip its rate limit again.

delay before next attempt (max)

Swipe sideways to see the whole figure.

Retry delay per attempt. The band is the jitter range; after the 10th failed attempt the event goes to the dead-letter queue, where an admin can inspect and retry it.

Events and channels#

Event	Who hears about it
`booking.requested`	host: a new request to approve
`booking.awaiting_payment`	driver: approved, pay within the hold
`booking.confirmed`	driver and host: email, push and inbox
`booking.checked_in · ending_soon · extended`	driver: session updates, a reminder 15 minutes before the end
`booking.completed`	driver: receipt; host: earnings
`booking.cancelled · rejected · expired`	the other party, plus refunds where due
`dispute.created · dispute.updated · payout.paid · payout.rejected`	the people involved
`auth.verification_code · auth.password_changed · account.deleted`	account security emails

Email delivery#

Swipe sideways to see the whole figure.

Resend reports what happened to each email through a signed webhook.

Bounced and complained addresses are suppressed, so we stop sending to them and protect the sending domain's reputation.
Verification codes are scrubbed from the stored outbox payload once sent; no secret sits in the database longer than needed.
Users choose which channels they want per category in notification preferences; security emails always go out.