Idempotent Message Handling
in Azure Service Bus

A customer gets charged twice. Two "your order has shipped" emails go out for one order. A ledger entry is written, then written again a few seconds later. When you trace it back, nothing was broken: the message was delivered twice because a consumer did the work, then crashed — or simply ran long — before it acknowledged the message, and the broker did exactly what it promises and redelivered. The fix is not a bigger broker or a retry tweak; it's idempotent message handling — designing the consumer so that processing the same message twice has the same effect as processing it once.

This is the single most common correctness bug we see in event-driven .NET systems, and it's usually met with two misconceptions: that Azure Service Bus duplicate detection already prevents it, or that switching the receive mode to something "safer" makes it go away. Neither is true. This article gives you a working framework for idempotent message handling in Azure Service Bus: why at-least-once delivery guarantees duplicates in the first place, the three levels of idempotency and when each applies, the transactional deduplication pattern that actually closes the race (with production .NET code), and the failure modes — including why "exactly-once" is not something you can buy.

Why At-Least-Once Delivery Guarantees Duplicates

Azure Service Bus delivers messages at least once. In the default PeekLock receive mode, a message is locked for your consumer, you process it, and only when you call CompleteMessageAsync does the broker remove it. If your process crashes after the side effect but before that acknowledgement — or the handler runs past the lock duration, or the Complete call itself times out — the lock expires and the message becomes visible again. The broker has no way to know whether you finished the work; from its side, an unacknowledged message is an unprocessed message, so it redelivers. That is at-least-once delivery working exactly as designed, not a fault.

The common rebuttal is: "but we enabled duplicate detection." That helps with a different problem. Service Bus duplicate detection keeps a history of MessageId values for a configurable time window and discards a second send of the same id — it protects you when a producer retries and publishes the same message twice. It does nothing about a consumer that processes a message and then fails to acknowledge it, because that is one send and multiple deliveries, not two sends. Send-side dedup and receive-side idempotency are different guarantees, and only the second one stops the double charge.

So the duplicate is inevitable, and the broker cannot remove it for you. The receiver has to be built so a repeat is harmless. This is a well-worn idea — Gregor Hohpe and Bobby Woolf named it the Idempotent Receiver in Enterprise Integration Patterns (Addison-Wesley, 2003) — but the interesting part is not the name; it's the handful of ways to implement it and the races that catch teams who implement it naively.

The Three Levels of Idempotent Message Handling

Not every message needs the same machinery. The cheapest correct option depends on what the handler actually does. We think about it as three escalating levels — pick the lowest one that holds for your operation, because each level up costs you a write, a constraint, or a stored record you now have to manage.

Level 0 is the one teams skip past too quickly. If you can express the side effect as an idempotent operation — an upsert keyed on the entity, a state transition that's a no-op when already in the target state — you need no deduplication infrastructure at all, and you avoid a whole class of retention and concurrency problems. Reach for Levels 1 and 2 only when the operation genuinely cannot be made naturally idempotent.

The Naive Dedup Check — and the Two Races It Loses

When a stable MessageId is available, the obvious Level-1 implementation is: check whether we've seen this id, and if not, do the work and record the id. It reads correctly and passes every test you run on a single thread. Here is that version:

// NAIVE: check-then-act. Two windows where a duplicate still slips through.
public class OrderMessageHandler
{
    private readonly IProcessedMessageStore _dedup;
    private readonly IOrderService _orders;

    public async Task HandleAsync(ServiceBusReceivedMessage message, CancellationToken ct)
    {
        // 1) Have we seen this message id before?
        if (await _dedup.HasProcessedAsync(message.MessageId, ct))
            return;

        var order = message.Body.ToObjectFromJson<PlaceOrder>();

        // 2) Perform the side effect.
        await _orders.PlaceAsync(order, ct);

        // 3) Record that we processed it.
        await _dedup.MarkProcessedAsync(message.MessageId, ct);
    }
}

This loses two races. First, the crash window: if the process dies between step 2 and step 3, the id is never recorded; the message is redelivered, the check at step 1 passes, and the order is placed a second time. Second, the concurrency window: if the message is delivered to two competing consumers at once — or the lock expires mid-handler and the broker hands the same message to another instance while the first is still running — both pass step 1 before either reaches step 3, and both place the order. The check and the act are two separate operations with a gap between them, and every duplicate lives in that gap.

The Production Pattern: Atomic Deduplication

The fix is to stop treating "record that we processed it" as a step that follows the work, and make it part of the same transaction as the work. Insert the dedup row and the business change together; put a unique index on the message id; let the database reject the second insert. There is no longer a window between check and act, because the check is the act — the unique constraint does the deduplication atomically, under whatever concurrency the database is handling.

// PRODUCTION: the dedup record and the business write commit in ONE transaction.
// A unique index on ProcessedMessage.MessageId makes the check atomic —
// there is no gap between "have we seen it" and "do the work".
public class OrderMessageHandler
{
    private readonly OrderDbContext _db;
    private readonly ILogger<OrderMessageHandler> _logger;

    public OrderMessageHandler(OrderDbContext db, ILogger<OrderMessageHandler> logger)
    {
        _db = db;
        _logger = logger;
    }

    public async Task HandleAsync(ServiceBusReceivedMessage message, CancellationToken ct)
    {
        var order = message.Body.ToObjectFromJson<PlaceOrder>();

        await using var tx = await _db.Database.BeginTransactionAsync(ct);
        try
        {
            // Claim the message id. If it's already there, the unique index
            // rejects the insert and we know the work was already committed.
            _db.ProcessedMessages.Add(new ProcessedMessage
            {
                MessageId = message.MessageId,
                ProcessedAtUtc = DateTime.UtcNow
            });

            // Same DbContext, same transaction as the dedup row.
            _db.Orders.Add(order.ToEntity());

            await _db.SaveChangesAsync(ct);   // throws on a duplicate MessageId
            await tx.CommitAsync(ct);
        }
        catch (DbUpdateException ex) when (IsUniqueViolation(ex))
        {
            // Duplicate delivery: the first pass already committed this work.
            await tx.RollbackAsync(ct);
            _logger.LogInformation(
                "Duplicate message {MessageId} ignored — already processed", message.MessageId);
        }
    }

    // SQL Server: 2627 (unique constraint) / 2601 (unique index).
    private static bool IsUniqueViolation(DbUpdateException ex) =>
        ex.InnerException is SqlException sql && (sql.Number == 2627 || sql.Number == 2601);
}

Three things make this correct where the naive version wasn't. The dedup row and the business write are the same atomic unit, so they either both land or neither does — the crash window is gone. The unique index, not application code, is the concurrency guard, so two simultaneous deliveries can't both win; one commits and the other takes the DbUpdateException path. And critically, a duplicate is handled by swallowing the violation and returning normally — which lets the processor go on to call CompleteMessageAsync and settle the message, so it stops redelivering. If instead the handler had thrown, the message would abandon and come back forever. Only genuine failures (a downstream outage, a real bug) should propagate and trigger a retry.

Business-Key Idempotency for Re-Published Events

Message-ID dedup assumes the same delivery always carries the same MessageId. That holds for broker redelivery, but not always for the source. If an upstream service retries its own processing and re-publishes the event, you may receive the same business intent under a brand-new message id — and a MessageId dedup store will happily treat it as new. When that's possible, deduplicate on the natural business key instead of the transport id:

// LEVEL 2: same intent may arrive under a NEW MessageId, so dedup on the
// business key. The unique constraint on (CustomerId, ExternalOrderRef)
// is the real guard; the read is only a fast path for the common case.
public async Task HandleAsync(PlaceOrder order, CancellationToken ct)
{
    var exists = await _db.Orders.AnyAsync(
        o => o.CustomerId == order.CustomerId && o.ExternalOrderRef == order.ExternalOrderRef, ct);
    if (exists)
        return; // this business intent is already recorded

    _db.Orders.Add(order.ToEntity());

    try
    {
        await _db.SaveChangesAsync(ct);
    }
    catch (DbUpdateException ex) when (IsUniqueViolation(ex))
    {
        // Lost the race to a concurrent delivery — the other one won, which is fine.
    }
}

The AnyAsync read is an optimization, not the safety mechanism. Under concurrency two handlers can both read "not there" and both try to insert; the unique constraint on the business key is what makes exactly one of them succeed. This is the rule that survives every version of this problem: the database constraint is the guarantee; the application-level check is only there to avoid throwing in the common case. Never rely on the read alone.

Choosing the Right Level

Put the decision in one place. Start at the top and stop at the first row that describes your handler — moving down the table only when the row above genuinely doesn't fit:

Tradeoffs and Failure Modes

This framework buys you effectively-once processing; it does not buy you exactly-once delivery, and it's worth being precise about what it doesn't cover.

• "Exactly-once" is a property of the effect, not the transport. You cannot atomically commit a database write and an acknowledgement to a separate broker in one step, so true exactly-once delivery across the two isn't available. Martin Kleppmann works through why in Designing Data-Intensive Applications (O'Reilly, 2017): the practical target is at-least-once delivery plus idempotent processing, which yields the same observable result as exactly-once without pretending the delivery guarantee exists.
• The atomicity trick needs a shared transaction. The Level-1 pattern works because the dedup row and the side effect live in the same database. If the side effect is an external call — charging a card, sending mail — it can't join that transaction, and you're back to check-then-act. Use the provider's idempotency key there, so the system that owns the side effect owns the dedup.
• Two side effects in one handler is outbox territory. If a handler both writes your database and calls an external service, no single dedup row makes the pair atomic. That's the transactional-outbox / saga problem, not the idempotency problem — see our piece on replacing batch generation with event-driven .NET for how that surrounding pipeline is structured.
• The dedup table needs a retention window. A ProcessedMessages table grows without bound. Prune it, but never below the maximum redelivery horizon — lock duration, the full retry-and-backoff schedule, and any manual dead-letter reprocessing — or a very late redelivery will read as brand new.
• Poison messages still dead-letter. Idempotency makes retries safe; it does not make a broken message succeed. After the max delivery count it moves to the dead-letter queue, and whatever reprocesses it later must go through the same idempotent path, or you've reopened the hole at the back door.

Idempotency is also the property that makes the ordered, session-based consumers from our Service Bus vs Event Hub comparison safe to retry, and it underpins the replayable, auditable pipelines described in architecting auditable backend systems. If you're deciding whether a broker belongs in your design at all, start there; if the broker is already in place, this is the pattern that keeps its redeliveries from corrupting your data. For the full shape of a system built this way, our real-time financial statements case study shows where idempotent handlers sit in the pipeline.

Doesn't Azure Service Bus duplicate detection already prevent duplicates?

Only on the send side. Duplicate detection keeps a history of MessageIds for a configurable window and discards a second send of the same id — it protects against a producer publishing twice. It does nothing about a consumer that processes a message and then fails to acknowledge it, which is one send and multiple deliveries. Preventing that duplicate side effect is the consumer's job.

What is the difference between at-least-once and exactly-once delivery?

At-least-once means the broker delivers a message one or more times — never lost, but possibly repeated after a consumer failure. Exactly-once delivery across a broker and an external side effect generally isn't achievable, because the acknowledgement and the side effect can't commit atomically together. What you can achieve is effectively-once processing: accept at-least-once delivery and make the handler idempotent.

Where should I store processed message IDs for deduplication?

In the same transactional store as the side effect, with a unique constraint on the message id. When the dedup record and the business write commit in one transaction, the unique index becomes the concurrency guard and there's no gap between checking and acting. A separate cache or store that commits independently reintroduces the exact race you're trying to close.

How long should I keep deduplication records?

Long enough to cover the maximum window a message could still be redelivered: lock duration, the full retry-and-backoff schedule, and any manual dead-letter reprocessing. Prune on a retention window rather than deleting a record the moment you process it — remove it too early and a late redelivery reads as new. A background job that deletes rows older than the redelivery horizon keeps the table bounded.

When to Bring in External Help

Duplicate-processing bugs are hard to catch in testing precisely because they only appear under the conditions tests don't reproduce: a crash mid-handler, a lock expiring under load, two consumers racing on the same message. They surface in production as a double charge or a duplicated ledger entry, and by then the bad data has to be found and unwound by hand. A short review of your message handlers against these failure modes — before they run at volume — is far cheaper than the cleanup.

A two-week Discovery Sprint reviews your event-driven .NET design for exactly this class of issue: delivery guarantees, deduplication strategy, and the transactional boundaries around each side effect.

Back to Insights