How We Built Datadog Experiments
Datadog, Thursday, September 10th, 2026
CUPED on percentiles, verifiable warehouse results and near real-time RUM metrics shorten result-to-decision time.
Datadog describes the engineering behind Datadog Experiments, built to shorten the gap between an experiment producing results and a team making a decision.
The technical contributions include applying CUPED variance reduction to percentile metrics rather than only means, producing results that can be independently verified against the data warehouse, and feeding near real-time RUM metrics into the analysis.
The percentile work is the most transferable part, since latency experiments almost always care about tail behavior that mean-based variance reduction does not help with.