Data-driven Experimentation Uncertainty 
O'Reilly on Data
MARCH 25, 2025
Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. Two big things: They bring the messiness of the real world into your system through unstructured data.
Rocket-Powered Data Science
FEBRUARY 15, 2023
Third, any commitment to a disruptive technology (including data-intensive and AI implementations) must start with a business strategy. Those F’s are: Fragility, Friction, and FUD (Fear, Uncertainty, Doubt). These changes may include requirements drift, data drift, model drift, or concept drift.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
O'Reilly on Data
MARCH 31, 2020
AI products are automated systems that collect and learn from data to make user-facing decisions. All you need to know for now is that machine learning uses statistical techniques to give computer systems the ability to “learn” by being trained on existing data. Machine learning adds uncertainty.
O'Reilly on Data
OCTOBER 13, 2020
From a technical perspective, it is entirely possible for ML systems to function on wildly different data. For example, you can ask an ML model to make an inference on data taken from a distribution very different from what it was trained on—but that, of course, results in unpredictable and often undesired performance. I/O validation.
The Unofficial Google Data Science Blog
DECEMBER 14, 2021
by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature.
The Unofficial Google Data Science Blog
APRIL 23, 2024
Crucially, it takes into account the uncertainty inherent in our experiments. Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. The data showed us that metrics are not exactly straight-line functions of parameters.
CIO Business Intelligence
NOVEMBER 15, 2022
Experiment with the “highly visible and highly hyped”: Gartner repeatedly pointed out that organisations that innovate during tough economic times “stay ahead of the pack”, with Mesaglio in particular calling for such experimentation to be public and visible.
Expert insights. Personalized for you.
168 
Let's personalize your content