Experimentation Testing Uncertainty 
O'Reilly on Data
MARCH 25, 2025
Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. What breaks your app in production isnt always what you tested for in dev! The way out?
Rocket-Powered Data Science
FEBRUARY 15, 2023
Those F’s are: Fragility, Friction, and FUD (Fear, Uncertainty, Doubt). encouraging and rewarding) a culture of experimentation across the organization. Keep it agile, with short design, develop, test, release, and feedback cycles: keep it lean, and build on incremental changes. Test early and often. Launch the chatbot.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
O'Reilly on Data
MARCH 31, 2020
Machine learning adds uncertainty. This has serious implications for software testing, versioning, deployment, and other core development processes. Underneath this uncertainty lies further uncertainty in the development process itself. Models within AI products change the same world they try to predict.
O'Reilly on Data
SEPTEMBER 15, 2020
Technical competence results in reduced risk and uncertainty. Results are typically achieved through a scientific process of discovery, exploration, and experimentation, and these processes are not always predictable. There’s a lot of overlap between these factors.
O'Reilly on Data
OCTOBER 13, 2020
In Bringing an AI Product to Market , we distinguished the debugging phase of product development from pre-deployment evaluation and testing. During testing and evaluation, application performance is important, but not critical to success. require not only disclosure, but also monitored testing. Debugging AI Products.
Domino Data Lab
MAY 15, 2019
Pete indicates, in both his November 2018 and Strata London talks, that ML requires a more experimental approach than traditional software engineering. It is more experimental because it is “an approach that involves learning from data instead of programmatically following a set of human rules.”
The Unofficial Google Data Science Blog
DECEMBER 14, 2021
by AMIR NAJMI & MUKUND SUNDARARAJAN Data science is about decision making under uncertainty. Some of that uncertainty is the result of statistical inference, i.e., using a finite sample of observations for estimation. But there are other kinds of uncertainty, at least as important, that are not statistical in nature.
Expert insights. Personalized for you.
184 
Let's personalize your content