Webhook Retries and Duplicate Events in OTP Workflows
Design an owned-app event workflow for duplicate, delayed and out-of-order notifications. Separate receipt from verification and keep secrets out of logs.
By PhoneBorn Team5 min read
Quick answer: design a webhook consumer so receiving the same event again does not repeat a sensitive action. Authenticate incoming notifications, save them durably, associate them with the correct message and challenge, and process them with explicit state rules. This is generic architecture for an application you own; it is not documentation of a PhoneBorn webhook or an undocumented endpoint.
Define the action that must happen only once
Begin with the business consequence, not the HTTP handler. Updating a message timeline is different from creating a verification challenge, marking an account verified or charging a customer. A duplicate notification should not accidentally repeat any of those consequences.
Consider an event that says an SMS was delivered. It may update delivery history, but it must not authenticate the person. Verification requires the application's separate challenge rules. Likewise, a callback about an earlier message must not mark a later resend as delivered just because both went to the same number.
Write these boundaries down before choosing a queue or database table. Otherwise, retry handling can become a patch around an undefined state machine.
Authenticate the delivery before trusting its fields
Follow the actual event provider's current signature-validation instructions and SDK behavior. Validate the raw payload in the way that provider requires; do not casually parse and reconstruct a signed body first. Keep signing secrets outside source code and out of callback URLs.
GitHub's webhook guidance recommends secrets, HTTPS and careful event selection for its service. Its headers and deadlines are GitHub-specific, so do not transplant them as if every provider uses the same contract.
For your own consumer, reject an invalid signature before letting user-controlled fields choose an account or trigger work. Also define how signing-key rotation and replay protection will be handled under the provider's documented model.
Separate durable receipt from background processing
Use a small ingress step that verifies the delivery and records enough information for safe later processing. Acknowledge according to the provider's contract after the necessary durable step succeeds. If storage fails, a successful response can hide an event that your system never retained.
A background worker can then perform slower transformations, updates and reconciliation. Do not make unrelated external calls a prerequisite for acknowledging every event unless you intentionally accept the resulting failure behavior.
The exact acknowledgment deadline varies by provider. For example, Telnyx's messaging documentation describes its event IDs, acknowledgment behavior and retries. Those details are an example of a documented contract, not a PhoneBorn feature assertion.
Use an event key and a separate business-operation key
An event identifier helps detect repeated delivery of the same notification. A business-operation identifier helps prevent the same intended action from being applied through several different events. They solve related but different problems.
| Identifier | Intended role |
|---|---|
| Provider event ID | Recognize another delivery of this event |
| Provider message ID | Attach observations to the same message |
| Local challenge ID | Identify the verification request |
| Business operation key | Prevent repeating a consequential update |
Enforce uniqueness where the durable write occurs. A check followed by an unprotected insert can race when two workers handle duplicates simultaneously. Design the transaction boundary so one outcome wins and the other worker can safely recognize that result.
Handle event order with explicit transition rules
Do not assume notifications arrive in the order events occurred. Stripe's webhook documentation explicitly warns about ordering and duplicate delivery for its events. That is a useful illustration of why a consumer should not depend on arrival order.
For your own messaging workflow, preserve both occurrence and receipt times, but do not use timestamps alone as a universal deduplication key. Several legitimate events can share a timestamp, and late observations can concern an already closed challenge.
Define allowed transitions. A late transport event can enrich a historical record without reopening an expired challenge or undoing a completed verification. If a provider documents correction events, model those deliberately rather than applying a simplistic rule that status can never change.
Test the failure windows with synthetic events
Create local fixtures containing no real OTPs or credentials. Deliver the same fixture twice; process two copies concurrently; delay one event; restart the worker between persistence and completion; and simulate a storage failure before acknowledgment.
For each case, identify the expected durable result. “No exception” is too weak an acceptance criterion. Confirm that one event record or one intended operation exists, that the challenge state remains correct, and that unfinished work can be retried.
A failed queue job should retain diagnostic identifiers without dumping its secret-bearing payload into a public error tracker. Use redacted fixtures when discussing defects with a vendor or contractor.
Reconcile gaps without confusing delivery and identity
Provide an operational view of pending, processed and failed event handling. If the provider supports retrieving authoritative message state, use its documented interface for reconciliation. Do not invent an endpoint or assume that another product offers equivalent retrieval.
A customer inbox can provide recipient-side evidence; it is not automatically a source of sender callback events. Review safe online SMS handling when working with received codes, and consult PhoneBorn facts for the scope of its number products.
Keep the final invariant simple: transport evidence may describe what happened to a message; only the correct application verification flow can establish that its challenge was completed.
Frequently asked questions
Does a duplicate event mean a duplicate SMS was sent?
Not necessarily. Repeated notification delivery and repeated message dispatch are different events.
Can a delivered callback mark a user verified?
It should not substitute for validating the user's challenge under your application's rules.
Are the event fields here PhoneBorn API fields?
No. This article describes generic design roles and cites providers as examples of their own contracts.