The Unofficial Google Data Science Blog

JULY 22, 2020

For this reason we don’t report uncertainty measures or statistical significance in the results of the simulation. From a Bayesian perspective, one can combine joint posterior samples for $E[Y_i | T_i=t, E_i=j]$ and $P(E_i=j)$, which provides a measure of uncertainty around the estimate. Henne, and Dan Sommerfield.

Experimentation 95

Experimentation Statistics Testing Knowledge Discovery 95

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

The result is that experimenters can’t afford to be sloppy about quantifying uncertainty. These typically result in smaller estimation uncertainty and tighter interval estimates. And since the metric average is different in each hour of day, this is a source of variation in measuring the experimental effect.

Experimentation 40

Experimentation Statistics Metrics Measurement 40