Studio journal
Latency budgets across service meshes
Written for teams practicing application analytics for microservices API experience.
Service meshes make policy elegant and timing murky. Each sidecar adds a courteous delay that rarely appears in the slide where someone promises “p95 under 200ms.” Application analytics has to account for those hops without turning every review into a blame session about infrastructure fashion.
Start from the customer moment
Pick the moment a human notices: button enabled, confirmation text, receipt email queued. Budget time backward from that moment. Mesh hops inherit leftovers; they do not get first claim on the budget simply because they are fashionable.
Measure the polite delays
Instrument ingress, sidecar, handler, and egress as separate spans when you can. If your vendor collapses them, annotate approximate hop cost from a controlled canary. In Service Signal Studio we ask learners to write the hop tax on the canvas in ink — visible, slightly uncomfortable, honest.
Negotiate with product, not only SRE
Product partners care whether checkout feels snappy in Busan evening peaks, not whether mTLS handshake percentiles look tidy. Bring one chart that ties mesh overhead to the customer moment. Leave the cipher suite debate for a different meeting.
A practical ceiling
If hop tax exceeds roughly a quarter of the experience budget, stop optimizing handlers in isolation. Either simplify the path, move work async, or renegotiate the promise. Pretending the mesh is free is how dashboards start lying.