David Menninger's Analyst Perspectives

MAY 20, 2025

We live in a time of uncertainty, not unpredictability. Yet those organizations that lack the technology to perform short planning cycles have little or no ability to measure trade-offs in the plans in which they are involved. Or, at least enduring the least amount of damage.

Forecasting 130

Forecasting Software Finance Uncertainty 130

Occam's Razor

APRIL 8, 2013

First, you figure out what you want to improve; then you create an experiment; then you run the experiment; then you measure the results and decide what to do. For each of them, write down the KPI you're measuring, and what that KPI should be for you to consider your efforts a success. Measure and decide what to do.

Metrics 157

Metrics KPI Analytics Key Performance Indicator 157

The Unofficial Google Data Science Blog

APRIL 23, 2024

the weight given to Likes in our video recommendation algorithm) while $Y$ is a vector of outcome measures such as different metrics of user experience (e.g., Crucially, it takes into account the uncertainty inherent in our experiments. Figure 2: Spreading measurements out makes estimates of model (slope of line) more accurate.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

The Unofficial Google Data Science Blog

APRIL 19, 2021

The numerical value of the signal became decoupled from the event it was measuring even as the ordinal value remained unchanged. bar{pi} (1 - bar{pi})$: This is the irreducible loss due to uncertainty. And users may start receiving a lot more spam! The further your predictions are from the global average the more you improve the loss.

Modeling 122

Modeling IT Metrics Testing 122

The Unofficial Google Data Science Blog

JULY 22, 2020

For this reason we don’t report uncertainty measures or statistical significance in the results of the simulation. From a Bayesian perspective, one can combine joint posterior samples for $E[Y_i | T_i=t, E_i=j]$ and $P(E_i=j)$, which provides a measure of uncertainty around the estimate. 2] Scott, Steven L.

Experimentation 96

Experimentation Statistics Testing Knowledge Discovery 96

The Unofficial Google Data Science Blog

MAY 31, 2016

It is important that we can measure the effect of these offline conversions as well. Panel studies make it possible to measure user behavior along with the exposure to ads and other online elements. Let's take a look at larger groups of individuals whose aggregate behavior we can measure. days or weeks).

Advertising 43

Advertising Testing Sales Statistics 43

O'Reilly on Data

DECEMBER 9, 2019

Because of this trifecta of errors, we need dynamic models that quantify the uncertainty inherent in our financial estimates and predictions. Practitioners in all social sciences, especially financial economics, use confidence intervals to quantify the uncertainty in their estimates and predictions.

Statistics 32

Statistics Uncertainty Risk Marketing 32