Average vs p95/p99 response time: why do we use percentiles?

Why the average misleads

Response times are not normally distributed. Most requests are fast, and a minority are much slower because of garbage collection pauses, cache misses, lock contention, cold connections or slow downstream calls. That gives a long right tail. The mean blends everything into one number that may not describe any real request.

A simple illustration: if 90 requests take 100 ms and 10 take 3,000 ms, the average is 390 ms. Nobody experienced 390 ms. Ninety users had a fast response, ten waited three seconds, and the average hides both facts. The reverse also happens: a handful of extreme outliers can drag the average up while the typical experience is fine.

What percentiles mean

• p50 (median): half of requests were at or below this time. The typical experience.
• p95: 95% of requests were at or below this time; 1 in 20 was slower.
• p99: 1 in 100 was slower. This is tail latency.
• max: a single worst sample; useful for spotting anomalies but too noisy for targets.

Why the tail matters more than it seems

A single page load or user journey often triggers many requests: API calls, assets, and in microservices, fan-out to several backends. The more requests a user action involves, the more likely at least one hits the slow tail. So a “rare” p99 becomes a common user experience. That is why mature teams set objectives on p95 and p99, not on the average.

Using percentiles well

Targets should combine percentiles with error rate and throughput, otherwise a system that fails fast can look great on latency.

// k6 thresholds example
export const options = {
  thresholds: {
    http_req_duration: ['p(95)<400', 'p(99)<1000'],
    http_req_failed: ['rate<0.01'],
  },
};

Practical points:

• Percentiles do not average. The mean of two servers’ p95 values is not the p95 of the combined traffic. Aggregate from raw data or histograms, which is how monitoring systems such as Prometheus compute them.
• Sample size matters. A p99 from 200 requests is based on two samples; run long enough for stable numbers.
• Look per endpoint. A global p95 can hide one slow, critical endpoint behind many fast health checks or static calls.
• Compare distributions between runs, not just one number; a shift in p50 and p99 tells different stories (general slowdown vs contention).

What interviewers look for

• A clear explanation of skewed distributions and why the mean misleads.
• Correct definition of p95 and p99.
• Why tail latency matters for real users, and thresholds that combine percentiles with error rate.