The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

Occam's Razor

APRIL 8, 2013

Remember that the raw number is not the only important part, we would also measure statistical significance. They might deal with uncertainty, but they're not random. The result? The properties with professional photography had 2-3 times the number of bookings! Airbnb had enough data points to be confident in their results.

Metrics 157

Metrics KPI Analytics Key Performance Indicator 157

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. But these are not usually amenable to A/B experimentation.

Experimentation 68

Experimentation Sales Metrics Measurement 68

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Yet, this challenge is not insurmountable. for what is and isn’t possible) to address these challenges.

Management 110

Management Machine Learning Experimentation Metrics 110

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

In this post we explore why some standard statistical techniques to reduce variance are often ineffective in this “data-rich, information-poor” realm. Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant.

Experimentation 40

Experimentation Statistics Metrics Measurement 40

IBM Big Data Hub

APRIL 18, 2024

LLMs like ChatGPT are trained on massive amounts of text data, allowing them to recognize patterns and statistical relationships within language. The AGI would need to handle uncertainty and make decisions with incomplete information. NLP techniques help them parse the nuances of human language, including grammar, syntax and context.

Cost-Benefit 89

Cost-Benefit Manufacturing Modeling Interactive 89

The Unofficial Google Data Science Blog

MAY 31, 2016

A geo experiment is an experiment where the experimental units are defined by geographic regions. Statistical power is traditionally given in terms of a probability function, but often a more intuitive way of describing power is by stating the expected precision of our estimates. They are non-overlapping geo-targetable regions.

Advertising 42

Advertising Testing Sales Statistics 42

The Unofficial Google Data Science Blog

OCTOBER 21, 2015

It is important to make clear distinctions among each of these, and to advance the state of knowledge through concerted observation, modeling and experimentation. Note also that this account does not involve ambiguity due to statistical uncertainty. We sliced and diced the experimental data in many many ways.

Slice and Dice 45

Slice and Dice Experimentation Data-driven Data Science 45