An endpoint that used to return in 200ms now takes two seconds, and nobody changed the code that handles it. Or it was always slow, and the team has been guessing at fixes for weeks — adding caching here, an index there, a retry policy somewhere else — with no measurable change in the metric that matters. Both situations have the same root cause: nobody has actually located the constraint. They're treating symptoms.
This article is a step-by-step playbook for finding what's actually slowing one specific .NET API endpoint, in order, before any remediation code gets written. It assumes ASP.NET Core on Azure, but the method applies to any .NET web API regardless of host.
Step 1 — Profile the Request, Not the App
"The app is slow" is not a diagnosable statement. A specific endpoint, under a specific concurrency level, with a specific payload shape is diagnosable. Start by isolating the one or two endpoints actually driving complaints — usually visible in Application Insights as the requests with the worst p95/p99 duration, not the worst average.
Average latency hides the problem. A request that's fast 95% of the time and catastrophic 5% of the time has an average that looks acceptable but a user experience that doesn't. Pull the distribution, not the mean, before doing anything else.
- In Application Insights, query the
requeststable for the specific operation, grouped by p50/p95/p99 over the time window where the complaint originated. - Confirm whether slowness correlates with request volume (a throughput problem) or is constant regardless of load (a fixed-cost problem — usually a single slow dependency call or a large payload).
- If Application Insights isn't wired up for this endpoint, that's the first fix — you cannot diagnose what you cannot see.
Step 2 — Separate I/O Wait From CPU Work
Once you know which requests are slow, the next question is where the time goes: waiting on something external, or actually computing something on the CPU. These have completely different fixes, and conflating them is the single most common mistake in performance investigations.
Application Insights' end-to-end transaction view shows this directly for distributed calls — database, HTTP dependencies, Service Bus. If the request duration is mostly dependency duration, you have an I/O-wait problem and the fix is in the dependency or in how it's called (parallelism, caching, connection pooling). If dependency duration is small relative to total request time, the time is being spent in your own code — and the fix is a CPU/ allocation profile, not a database tune-up.
// Don't guess — confirm where time is spent before optimizing anything
public async Task<IActionResult> GetPortfolioSummary(int accountId, CancellationToken ct)
{
using var activity = _telemetry.StartActivity("GetPortfolioSummary");
var account = await _accountService.GetAsync(accountId, ct); // dependency span
var holdings = await _holdingsService.GetAsync(accountId, ct); // dependency span
var summary = _summaryCalculator.Build(account, holdings); // in-process CPU work — separate span if slow
return Ok(summary);
}
Wrapping in-process computation in its own tracked activity, separate from the dependency calls, is what makes this distinction visible in the trace instead of a guess.
Step 3 — EF Core Query Patterns That Lie About Cost
If the dependency span pointing at the database is the dominant cost, the next step is
capturing the actual generated SQL — not the LINQ that produced it. EF Core's logging
(ILoggerFactory with the Microsoft.EntityFrameworkCore.Database.Command
category) or SQL Server Profiler will show you what's really being sent.
The query patterns that look fine in code but are expensive in practice:
- N+1 from lazy loading — a list query followed by per-item navigation property access, issuing one query per row instead of one query total.
- Over-fetching —
.ToListAsync()on a full entity when the endpoint only needs three columns; the cost is in materialization and network transfer, not the query plan itself. - Untracked change-tracking overhead — read-only endpoints that don't call
.AsNoTracking()pay for EF Core's change-detection machinery on data that's never going to be saved. - Client-side evaluation — a filter that can't be translated to SQL silently falls back to pulling the full table into memory and filtering in .NET.
// ✗ Tracked, over-fetched, and lazy-loads OrderItems per order
var orders = await _db.Orders
.Where(o => o.AccountId == accountId)
.ToListAsync(ct);
var total = orders.Sum(o => o.OrderItems.Sum(i => i.Price)); // N+1 on access
// ✓ Projected straight to a scalar — one query, no entity materialization
var total = await _db.Orders
.Where(o => o.AccountId == accountId)
.SelectMany(o => o.OrderItems)
.SumAsync(i => i.Price, ct);
// ✗ Read-only list query that still pays for change-tracking on every row
var orders = await _db.Orders
.Where(o => o.AccountId == accountId)
.ToListAsync(ct);
// ✓ AsNoTracking matters here — the result is a list of tracked-by-default entities
var orders = await _db.Orders
.Where(o => o.AccountId == accountId)
.AsNoTracking()
.ToListAsync(ct);
Step 4 — GC Pressure and Allocation Hotspots
If dependency spans are fast and the time is genuinely spent in your own code, the next
suspect is allocation pressure. dotnet-counters attached to the running process
under realistic load shows % Time in GC and per-generation collection rate in
real time without restarting the process.
Sustained % Time in GC above roughly 10-15% under load is worth investigating.
The usual culprits in API code are large allocations on hot paths: building large strings
with + instead of StringBuilder or string.Create,
materializing intermediate LINQ collections that get discarded immediately, and serializing
large response payloads without streaming. dotnet-trace with the allocation
provider will point at the specific call stacks responsible if the counter alone isn't
conclusive.
Step 5 — When the Bottleneck Is Downstream (and How to Confirm)
Sometimes none of the above account for the latency, because the actual constraint is a downstream dependency your API calls — a third-party API, a partner system, or another internal service — that is itself slow or rate-limiting you. This shows up in Application Insights as a dependency span that dominates total request duration with no corresponding spike in your own CPU or GC metrics.
Confirming this is straightforward: if you can reproduce the slow response by calling the downstream dependency directly (Postman, a small console harness) at the same time of day and concurrency, the constraint is confirmed and outside your control to fix directly. The remediation options then shift to architecture — caching, circuit breakers, async processing via a queue instead of a synchronous call in the request path — rather than code-level optimization.
Putting the Five Steps Together
Run the steps in order, and stop as soon as one of them confirms a candidate constraint — don't run all five on every investigation. Most slow endpoints resolve in step 2 or 3.
| Step | Tool | Confirms |
|---|---|---|
| 1. Profile the request | Application Insights (p95/p99) | Which endpoint, under what load |
| 2. I/O vs. CPU | App Insights transaction view | Dependency wait vs. in-process work |
| 3. EF Core query shape | EF Core logging / SQL Profiler | N+1, over-fetch, untracked reads |
| 4. GC pressure | dotnet-counters / dotnet-trace | Allocation-driven slowness |
| 5. Downstream dependency | Direct reproduction outside your API | Constraint is outside your codebase |
What This Method Doesn't Solve
This playbook diagnoses a single endpoint's latency. It does not replace a full architecture review when an entire system is undersized for its load, and it does not substitute for capacity planning ahead of a known traffic event. If every endpoint is slow simultaneously, the question is usually infrastructure-level (database tier, network path, regional placement) rather than per-endpoint — see our broader .NET performance bottleneck diagnosis framework for that wider investigation.
Frequently Asked Questions
- Why is my .NET API faster locally than in production?
- Local testing rarely reproduces production concurrency, production data volumes, or network hops to dependencies in the same region. A query that returns in milliseconds against a 200-row local database can take seconds against a production table with millions of rows and a different execution plan. Concurrency-dependent issues are often invisible below a certain number of simultaneous requests, which local testing almost never reaches.
- Is async/await making my API slower?
- Async/await itself adds negligible overhead. The actual problem is usually the opposite: synchronous blocking calls (
.Result,.Wait()) mixed into an otherwise async call chain, which holds thread-pool threads hostage and starves the pool under load. If async code feels slow, look for a sync-over-async call site before suspecting the async model itself. - When is EF Core the problem vs. the database?
- EF Core is the problem when the query it generates is structurally wrong for the data shape — N+1 patterns, missing
.Include()calls, or over-fetched projections. The database is the problem when even a correctly-shaped query is slow because of a missing index or lock contention. Capture the generated SQL and run it directly against the database outside the application — if it's still slow, the database is the constraint. - How do I tell if it's GC pressure?
- Use
dotnet-countersto watch the per-generation collection rate and% Time in GCunder realistic load. Sustained values above roughly 10-15% are worth investigating. The usual cause is large or frequent allocations on the request path — unnecessary LINQ materialization, inefficient string building, or large response payloads built entirely in memory.
When to Bring in External Help
This method works well when a team has the bandwidth to run it end-to-end on one endpoint at a time. It works less well when there are a dozen slow endpoints, no existing telemetry, and a backlog of feature work competing for the same engineers' attention — which is the situation we're usually called into.
A two-week Discovery Sprint applies this exact method across your highest-impact endpoints, with telemetry wired up where it's missing, and delivers a ranked list of confirmed constraints and a remediation roadmap. For a real example of what that diagnosis looks like applied to a system under genuine production load, see the real-time financial statements case study.