Avishai Ish-Shalom:

To sum up: Variance is the enemy of performance and the source of much of the latency we encounter when using software.

To keep latency to a minimum:

• As a rule of thumb, target utilization below 75%,
• Steer slower workloads to paths with lower utilization,
• Limit variance as much as possible when utilization is high,
• Implement backpressure in systems where it is not built-in,
• Use throttling and load shedding to reduce pressure on downstream queues.