Complex integration design
Proposed architecture · client workflow to be documented
The workflow finished. Did the business get the right result?
It started with…
A workflow can succeed partially, receive a late response or miss an event. A green dashboard alone does not explain discrepancies between systems.
The proposed workflow
Periodic reconciliation across Stripe, Airtable and Sage references, with pagination, consistency checks, assigned exceptions and recovery from an identified step.
- Stripe
Verified event
Protected receiver and transactional ledger
- Zapier
Coordination
Airtable for tracking, code for complex rules
- Sage
Action through an adapter
Result reference or an exception to handle
Design diagram to adapt to the actual Zap and Sage edition.
What we set out to change
Give operations an actionable discrepancy list and a controlled way to resolve it.
How it is builtData model, code, controls and team ownership
The user journey
- 01
Collect references over an overlapping time window.
- 02
Compare states and classify discrepancies.
- 03
Reconcile or approve recovery, then retain its result.
Data and hand-off structure
| Entity / table | Key fields | Relationships |
|---|---|---|
| Reconciliation runs | Period · cursor · counts · status | A saved checkpoint for each run |
| Discrepancies | Source reference · expected state · observed state | One business issue, potentially several events |
| Recovery decisions | Owner · reason · approval · outcome | Trace who authorised the next step |
Technical implementation
- Paginate APIs, respect their limits and save a checkpoint.
- Use an overlapping window and stable keys to include late events.
- Separate transport failures, business rejections and unknown remote outcomes; each needs a different next step.
Engineering decision
Choose recovery from what the destination may have done
Proposed recovery policy. A timeout after sending requires reconciliation; a rejected business operation goes to review. Only a failure known to occur before sending receives bounded exponential backoff with jitter, respecting Retry-After. Long waits are deferred. Persisted scheduling, checkpoint advancement and destination-specific error classification remain outside the pure function.
type Failure = | { kind: "business-rejection"; code: string } | { kind: "outcome-unknown" } | { kind: "not-sent"; retryAfterMs?: number };export function recoveryPolicy( failure: Failure, attempt: number, random = Math.random,) { if (failure.kind === "business-rejection") { return { next: "human-review", code: failure.code } as const; } if (failure.kind === "outcome-unknown") { return { next: "reconcile-before-write" } as const; } if (!Number.isSafeInteger(attempt) || attempt < 0 || attempt >= 6) { return { next: "escalate" } as const; } const ceiling = Math.min(60_000, 1_000 * 2 ** attempt); const jitter = Math.floor(Math.max(0, Math.min(1, random())) * ceiling); const retryAfter = failure.retryAfterMs ?? 0; if (!Number.isFinite(retryAfter) || retryAfter < 0) { return { next: "escalate" } as const; } const delayMs = Math.max(retryAfter, jitter); return { next: delayMs > 60_000 ? "defer" : "schedule-retry", delayMs, } as const;}// A response timeout is NOT evidence that no remote write occurred.// Persist the decision; do not hold a Zap step open with a long sleep.Adapted implementation excerpt
Controls and boundaries
- Keep card data out of Airtable; retain only required references and business information.
- Sensitive decisions and writes require explicit permissions and approval rules.
- The Sage connector and recovery capabilities depend on the chosen edition and plan.
Who can contribute
- Operations: reference data, approved rules and exception handling.
- Builders: documented Zapier steps and mappings within their agreed scope.
- Developers: API contracts, custom connectors, tests and versioning.
What supports this case
Architecture example developed to explain the approach. No client Zap export or run history has yet been reviewed; no production outcome is claimed.