Automating a Manual Back-Office Process:
Where to Start

When a team decides to automate a manual back-office process, it usually picks the one that hurts most: the month-end close, the weekly file that six people re-key, the refresh that takes over a weekend. That is a reasonable instinct and a risky first choice. The process that hurts most is often the one with the most exceptions, the least written-down rules and the highest cost when a single item is wrong. That is why it hurts.

Two things decide whether the project works. The first is whether the process is ready to be automated at all, which is a question about its rules and its data, not about technology. The second is how you switch over. Cutting a process over in one weekend, on the strength of a demo, is how a team learns about the exceptions in production.

This article gives you a four-test scorecard for the first question and a side-by-side run, with working .NET code, for the second. It is written for the CTO, engineering lead or operations owner who is scoping the project and has to say what to start with, what it will save, and how to avoid a bad first month.

Before You Automate a Manual Back-Office Process, Score It

Most readiness advice is a list of things to "consider". A list doesn't tell you what to do on Monday. What helps is a small number of tests, each with an observable answer. Here are the four we use. This is our own scorecard, not a standard, and the wording of each level is our judgement.

Score the first three tests and add them up, out of 6. The fourth test isn't summed, because it doesn't say whether the process can be automated. It says how carefully. Read the total like this:

• 5 to 6: automate the whole flow, and run it side by side first (below).
• 3 to 4: automate the routine path and keep a human queue for the exceptions. Most processes land here, and it's a good outcome.
• 2 or less, or a red on rule stability: not yet. Write the rules down and standardise the input first. Software that encodes rules two people disagree on doesn't remove the disagreement. It hides it inside code that nobody wants to change.

A red on cost of error doesn't block the project. It changes the design: a person approves the item before anything irreversible happens, the pricing or payment fields get explicit validation, and the side-by-side run gets longer. Automating the mechanical work around a decision, and leaving the decision with a person, is still automation.

How to Fill In the Scorecard in a Week

Don't score in a meeting. People overrate how regular their own process is, and the exceptions are the part they've stopped noticing. Get evidence instead:

• Rule stability: sit with the operator and ask them to work through ten recent items aloud. Every time they say "it depends", write down what it depends on. If the answer is a rule you can state, that's amber or green. If it's "I just know", that's red.
• Exceptions: take a sample of recent items and tag each one as routine or exception, and for each exception, note what told the operator. That last column is your future validation rule. If it's blank, the machine can't recognise that exception.
• Data access: ask for the actual input files and the actual destination screen, not a description. Whether the target has an import or an API is a fact you can find out in an hour.
• Cost of an error: ask what happened the last time an item was wrong. The answer tells you more than a risk workshop.

While you're there, record the size of the prize: person-hours per cycle, and cycles per year. Multiply them. That number, and not the technology, is the business case. Then look for the single step that takes the most of those hours and scores best. Start there, not with the whole process. Microsoft's Strangler Fig pattern page makes the same argument for software migrations: replace functionality in stages while the old system keeps working, because replacing a complex system in one go is hard. The page's own caveat applies here too: for a small process, replacing the lot in one go can be the simpler route.

What the Scorecard Says About a Real Case

We didn't have this table when we built the system in our retail catalog automation case study, so read this as a retrospective. A European retailer's catalog refresh took three to four days of work by fifty operations staff, run about three times a year. Scored today:

• Rule stability: amber. Every vendor sent a different file format and changed it without notice. Two earlier automation attempts had failed. One pipeline broke each time a vendor changed a format, which is exactly this problem. A middleware platform was too slow to finish inside the weekend window. The lesson wasn't "don't automate". It was to map each vendor to one canonical format and treat the vendor differences as data the system had to absorb.
• Exceptions: amber. The operators' judgement was mostly "this field is missing or this price is wrong". Once written as validation rules, each rejected item carried a machine-readable reason in a dead-letter queue, so an exception became a queue item, not a mystery.
• Data access: green. Feeds arrived as files. The pipeline sat as a layer on top of the existing ERP, so nothing had to be replaced.
• Cost of an error: red. A wrong price reaching the live catalog created consumer-facing and regulatory exposure. That is why the pricing fields got explicit validation and nothing was assumed correct from the source feed.

That adds up to 4 out of 6, which is the "routine path plus queue" band, with the red on cost of error setting how much care the design needs. The published result was a refresh that fell from three to four days of fifty people's work to under three hours, and about 1,200 to 1,600 person-hours per cycle handed back. The design behind it is in the case study. What this article adds is the order of the questions to ask first. If the process behind yours is a scheduled bulk job that only changes a little each cycle, our article on replacing batch jobs with event-driven .NET covers that different problem.

Run Old and New Side by Side Before You Switch

The scorecard tells you whether to start. It doesn't tell you when the automation is right, and a demo on last month's clean data won't either. The pattern for that is Sam Newman's Parallel Run, from Monolith to Microservices (2019). As Zalando's engineers describe it, both implementations run on the same input, the old one stays the source of truth, and the results are compared until the new one has earned the switch. GitHub used the same idea to rewrite its permissions code: its Scientist library runs the old code path as the control and the new one as the candidate, always returns the control's result to the user, and records any difference.

Both sources are about software, and both compare a request at a time. We add three rules for a back-office process: compare the business result item by item, not in total; switch one lane at a time, where a lane is a supplier, a region or a document type; and count consecutive clean cycles, not days. Here is the tempting version first:

// NAIVE: "the totals match, so it's right."
static bool NaiveTotalsMatch(
    IReadOnlyDictionary<string, ItemResult> manual,
    IReadOnlyDictionary<string, ItemResult> automated) =>
    manual.Count == automated.Count &&
    manual.Values.Sum(r => r.Amount) == automated.Values.Sum(r => r.Amount);

It fails as soon as two errors point in opposite directions: one item 10 too high and another 10 too low give the same total as two correct items. A wrong decision with the right amount passes as well. This version compares each item and says what differs:

/// <summary>What one lane (the people, or the automation) produced for one item.</summary>
public sealed record ItemResult(string ItemId, string Decision, decimal Amount);

public enum DifferenceKind { OnlyInManual, OnlyInAutomated, DecisionDiffers, AmountDiffers }
public sealed record Difference(string ItemId, DifferenceKind Kind, string Detail);

public sealed record ShadowReport(int Compared, IReadOnlyList<Difference> Differences)
{
    public bool IsClean => Differences.Count == 0;
}

public static class ShadowRun
{
    /// <summary>Item-by-item comparison. People stay the source of truth: nothing
    /// here writes to the destination system.</summary>
    public static ShadowReport Compare(
        IReadOnlyDictionary<string, ItemResult> manual,
        IReadOnlyDictionary<string, ItemResult> automated,
        CancellationToken ct = default)
    {
        var differences = new List<Difference>();

        foreach (var (id, m) in manual.OrderBy(kv => kv.Key, StringComparer.Ordinal))
        {
            ct.ThrowIfCancellationRequested();

            if (!automated.TryGetValue(id, out var a))
                differences.Add(new(id, DifferenceKind.OnlyInManual, "automation produced nothing"));
            else if (a.Decision != m.Decision)
                differences.Add(new(id, DifferenceKind.DecisionDiffers, $"{m.Decision} vs {a.Decision}"));
            else if (a.Amount != m.Amount)
                differences.Add(new(id, DifferenceKind.AmountDiffers, $"{m.Amount} vs {a.Amount}"));
        }

        differences.AddRange(automated.Keys.Except(manual.Keys)
            .OrderBy(id => id, StringComparer.Ordinal)
            .Select(id => new Difference(id, DifferenceKind.OnlyInAutomated, "no manual result to compare")));

        return new ShadowReport(manual.Count, differences);
    }
}

/// <summary>Per-lane gate: live only after N consecutive cycles with zero differences.</summary>
public sealed class CutoverGate(int requiredCleanCycles)
{
    private int _cleanCycles;

    public bool Record(ShadowReport report)
    {
        _cleanCycles = report.IsClean ? _cleanCycles + 1 : 0;
        return _cleanCycles >= requiredCleanCycles;
    }
}

We ran both versions on four sample items in which one amount is 10 too high, another is 10 too low and one decision differs. The naive check printed True. The comparison reported all three items, each with its difference, and the gate stayed closed until two consecutive clean cycles had been recorded. The differences the report lists are the input to your next step: each is either a bug in the automation, a rule nobody wrote down, or, now and then, a mistake by a person that nobody had noticed.

Switching One Lane at a Time

Keep one CutoverGate per lane. When a lane's gate opens, that lane goes live and the people stop producing its result, while the other lanes carry on in shadow. If a live lane later produces a bad item, the switch flips back for that lane only. Choose the lane order on purpose: start with the one where a wrong item is cheapest, and save the red-cost lane for last, when you trust the comparison. Two cycles is a starting point, not a standard. Raise it when a wrong item is expensive, and don't let a lane go live if its clean cycles never contained a real exception, because a lane that hasn't met one hasn't been tested on the part that matters.

What This Approach Doesn't Solve

• Shadow mode costs effort. Someone has to capture the manual result in a comparable form, and for a while the team does the work twice. Zalando's authors list the setup effort and the extra load among the drawbacks of a parallel run. Budget the double running as part of the project.
• The comparison is only as good as the manual result. If people's own output is inconsistent, the report fills with noise. That is a rule-stability problem the scorecard should have flagged.
• Side effects must stay off in shadow. The automation must not send the email, post the payment or update the record while it is only being compared. Design the write step so that it can be switched off per lane, and make it safe to run twice. Our article on idempotent message handling on Azure Service Bus covers the second half of that.
• Once people are out of the loop, silent failure is the new risk. A pipeline that drops one bad item in fifty thousand is worse than the manual process it replaced, because nobody is watching. Every outcome needs a log entry, and every rejection needs a reason and an owner.
• The scorecard is a judgement aid. Two evaluators can score the same process a point apart. Its value is that the disagreement is about a specific test and not about a feeling.

One condition where we'd take a different route: a process that runs once a year, with a red on cost of error and only a few hours of work in it. The side-by-side run costs more than the process does, and a checklist plus a second reviewer is the better investment. Automate when the volume, the error cost or the delay it causes justifies the machinery, not by default.

How do I decide which manual back-office process to automate first?

Score each candidate on four tests: whether its rules are stable and can be written down, whether the exceptions can be recognised by a rule, whether the input can be read and the output written by a program, and what one wrong item costs. Start with the step that scores best and also consumes the most staff time. Do not start with the process that hurts most if it scores badly, because automating an unstable process encodes the disagreement instead of removing it.

What makes a back-office process a bad candidate for automation?

Three things: rules that only exist in one person's judgement or change every cycle, exceptions that can only be spotted by a person reading the item, and inputs that arrive as paper or free text with no reliable structure. A high cost of error does not rule a process out, but it means a human approval step and a longer side-by-side run. In those cases, automate the mechanical steps around the decision and leave the decision with a person.

How do you run an automated process alongside the manual one before switching?

Let people keep producing the real result, run the automation on the same inputs without letting it write to the destination system, and compare the two item by item, not by totals. Totals can match while individual items are wrong in opposite directions. Switch one lane at a time, such as one supplier or one category, and only after it has produced a set number of consecutive cycles with no unexplained differences.

How long should the side-by-side run last?

Count cycles, not days. There is no standard number. A reasonable starting rule is two consecutive cycles with no unexplained differences, including at least one cycle that contained real exceptions. Raise it when a wrong item is costly and a lane runs rarely. A lane that has never produced an exception has not been tested on the part that matters.

How do I build the business case for automating a manual process?

Measure person-hours per cycle and cycles per year, then add the cost of rework and of the people being unavailable for other work while the cycle runs. In one of our retail case studies, a catalog refresh took roughly 1,200 to 1,600 person-hours per cycle and ran three times a year, which is roughly 3,600 to 4,800 staff-hours annually before counting rework. Count what the faster cycle enables too: that client ran six refreshes in the first year after deployment where it had run three.

When to Bring in External Help

You can run the scorecard yourself: it takes a week of an operator's time and a sceptical engineer. Outside help pays off when nobody on the team has built the comparison and cutover machinery before, when the process touches several systems nobody owns end to end, or when the red on cost of error means a mistake is expensive. Our process automation work follows the order in this article: map the steps, separate judgement from friction, automate the friction, and prove it side by side.

If your process scores 3 or higher, our Discovery Sprint is a two-week diagnostic that maps the process, scores it and writes down the automation path, including which lane goes first and what the side-by-side run has to prove.

Back to Insights