CIO Business Intelligence

NOVEMBER 13, 2023

Reimagination of business processes sits at the core of digital transformation, and so, by definition, digital transformation challenges the status quo, throwing we-have-always-done-it-this-way sentiment out of the window. Leaders must clearly define what they want to achieve through digital transformation and how they plan to do it.

Digital Transformation 143

Digital Transformation Cost-Benefit Experimentation Measurement 143

DataKitchen

OCTOBER 5, 2021

DataOps enables: Rapid experimentation and innovation for the fastest delivery of new insights to customers. Clear measurement and monitoring of results. Many people who work with data have a narrow definition of being “done.” Definition of “done” means “it worked for me”. “I Measure success. Low error rates.

Testing 154

Testing Dashboards Measurement Experimentation 154

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times. What are you measuring?

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

CIO Business Intelligence

JULY 7, 2023

A security-by-design culture incorporates security measures deeply into the design and development of systems, rather than treating them as an afterthought. Other experts agreed, and provided additional guidance: “Improving the experience of employees while maintaining security can be tricky. Caution is king, however.

Measurement 122

Measurement Experimentation Risk Enterprise 122

Occam's Razor

SEPTEMBER 19, 2011

This: You understand all the environmental variables currently in play, you carefully choose more than one group of "like type" subjects, you expose them to a different mix of media, measure differences in outcomes, prove / disprove your hypothesis (DO FACEBOOK NOW!!!), Measuring Incrementality: Controlled Experiments to the Rescue!

Measurement 120

Measurement Advertising Testing Marketing 120

Rocket-Powered Data Science

JULY 7, 2019

Definitions of terminology frequently seen and used in discussions of emerging digital technologies. Examples: (1-3) All those applications shown in the definition of Machine Learning. (4) Example applications: (1) High-definition and 3D video. (2) Career Relevance. NOTE: This page is a WIP = Work In Progress.). Industry 4.0

Internet of Things 157

Internet of Things Machine Learning Manufacturing IoT 157

CIO Business Intelligence

JUNE 11, 2024

A 1958 Harvard Business Review article coined the term information technology, focusing their definition on rapidly processing large amounts of information, using statistical and mathematical methods in decision-making, and simulating higher order thinking through applications.

IT 136

IT Digital Transformation Broadcasting Business Objectives 136

Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers. Counter claims?

Experimentation 103

Experimentation Testing Optimization Measurement 103

DataKitchen

APRIL 13, 2021

Because it is such a new category, both overly narrow and overly broad definitions of DataOps abound. DataOps requires that teams measure their analytic processes in order to see how they are improving over time. However, the rush to rebrand existing products with a DataOps message has created some marketplace confusion. Azure DevOps.

Testing 304

Testing Machine Learning Consulting Data Quality 304

Domino Data Lab

NOVEMBER 14, 2019

This Domino Data Science Field Note covers a proposed definition of interpretability and distilled overview of the PDR framework. James Murdoch, Chandan Singh, Karl Kumber, and Reza Abbasi-Asi’s recent paper, “Definitions, methods, and applications in interpretable machine learning” Introduction. Yet, Bin Yu, W.

Modeling 71

Modeling Machine Learning Data Science Experimentation 71

DataRobot

JUNE 17, 2021

Personally, I think there are definitely horizons in math, biology, and physics that we may never reach… but maybe AI can change that. Traditionally, science has advanced in many cases by having brilliant researchers compete different hypotheses to explain experimental data, and then design experiments to measure which is correct.

Modeling 52

Modeling Experimentation Machine Learning Measurement 52

O'Reilly on Data

OCTOBER 13, 2020

This distinction assumes a slightly different definition of debugging than is often used in software development. Emmanuel Ameisen once again provides a useful and concise definition of model monitoring as a way to “track the health of a system. While useful, these constructs are not beyond criticism.

Management 251

Management Machine Learning Metrics Modeling 251

datapine

FEBRUARY 4, 2020

Management thinker Peter Drucker once stated, “if you can’t measure it, you can’t improve it” – and he couldn’t be more right. But first, we will start with a basic definition and some tips on creating these kinds of reports. To know if you are successful, you first need to define success and track it. Let’s dig deeper.

Reporting 280

Reporting Marketing Advertising Metrics 280

Domino Data Lab

SEPTEMBER 10, 2020

This means many projects get stuck in endless research and experimentation. Problem statements should ultimately: Include a problem definition. Identify metrics that measure this variability. Problem Definition: I want to save costs by optimizing staffing rosters in my store without sacrificing customer experience.

Modeling 101

Modeling Data Science IT Metrics 101

Domino Data Lab

AUGUST 6, 2021

The early definition now accepted as standard in the industry is: “The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports.” The genesis of Model Risk Management was in the banking industry. Model implementation.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

Data Science and Beyond

APRIL 5, 2021

The initial covid-19 lockdown provided me with extra free time to make the measurement and offsetting of Automattic’s emissions from data centre power use happen. My main "day job" focus in 2020 was on being the tech lead for Automattic’s new experimentation platform (ExPlat). Technical work.

Experimentation 61

Experimentation Testing Publishing Data Collection 61

The Unofficial Google Data Science Blog

DECEMBER 14, 2021

In this post, we discuss three types of uncertainty: Statistical uncertainty : the gap between the estimand , the unobserved property of the population we wish to measure, and an estimate of it from observed data. Representational uncertainty : the gap between the desired meaning of some measure and its actual meaning.

Uncertainty 56

Uncertainty Statistics Measurement Cost-Benefit 56

Data Science and Beyond

DECEMBER 23, 2018

The book focuses on randomised controlled trials and well-defined interventions as the basis of causal inference from both experimental and observational data. As the authors show, even with randomised experiments, the analysis often requires using observational causal inference tools due to factors like selection and measurement biases.

Deep Learning 103

Deep Learning Experimentation Data Science Data-driven 103

Corinium

APRIL 25, 2019

Yes, definitely! For example, P&C insurance strives to understand its customers and households better through data, to provide better customer service and anticipate insurance needs, as well as accurately measure risks. It definitely depends on the type of data, no one method is always better than the other. Governance 101.

Insurance 150

Insurance Risk IoT Cost-Benefit 150

Occam's Razor

AUGUST 22, 2011

Key To Your Digital Success: Web Analytics Measurement Model. " Measuring Incrementality: Controlled Experiments to the Rescue! Barriers To An Effective Web Measurement Strategy [+ Solutions!]. Measuring Online Engagement: What Role Does Web Analytics Play? "Engagement" How Do I Measure Success?

KPI 125

KPI Metrics Measurement ROI 125

The Unofficial Google Data Science Blog

JANUARY 14, 2016

Unlike experimentation in some other areas, LSOS experiments present a surprising challenge to statisticians — even though we operate in the realm of “big data”, the statistical uncertainty in our experiments can be substantial. We must therefore maintain statistical rigor in quantifying experimental uncertainty.

Experimentation 41

Experimentation Statistics Metrics Measurement 41

Occam's Razor

JULY 18, 2011

You just have to have the right mental model (see Seth Godin above) and you have to… wait for it… wait for it… measure everything you do! For everything you do it is important to measure your effectiveness of all three phases of your effort: Acquisition. You’re trying to measure how well you are doing to: Send emails.

Metrics 138

Metrics Marketing Measurement Cost-Benefit 138

Ontotext

MAY 18, 2020

Ever since Hippocrates founded his school of medicine in ancient Greece some 2,500 years ago, writes Hannah Fry in her book Hello World: Being Human in the Age of Algorithms , what has been fundamental to healthcare (as she calls it “the fight to keep us healthy”) was observation, experimentation and the analysis of data. Certainly not!

Knowledge Discovery 72

Knowledge Discovery Experimentation Data-driven Metadata 72

Occam's Razor

NOVEMBER 12, 2006

Mostly because short term goals drive a lot of what we do and if you are selling something on your website then it only seems to make logical sense that we measure conversion rate and get it up as high as we can as fast as we can. So measure Bounce Rate of your website. Another reason is that we over estimate what is possible.

Measurement 99

Measurement Analytics Marketing Optimization 99

Occam's Razor

APRIL 8, 2013

First, you figure out what you want to improve; then you create an experiment; then you run the experiment; then you measure the results and decide what to do. For each of them, write down the KPI you're measuring, and what that KPI should be for you to consider your efforts a success. Measure and decide what to do.

Metrics 157

Metrics KPI Analytics Key Performance Indicator 157

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. These measurement-obsessed companies have an advantage when it comes to AI.

Management 110

Management Machine Learning Experimentation Metrics 110

Ontotext

MARCH 29, 2021

More than 10 years ago we even had experimental support of what we called triplesets , which allowed the association of metadata to sets of statements. we did practical modeling of various approaches and measured their performance. It is our hope that RDF-star is widely adopted and will be the definitive standard.

Metadata 59

Metadata IT Modeling Experimentation 59

Ontotext

APRIL 3, 2024

So, this is a big driver for the outcome because when you are saving money for the business, you can measure it and see its value. Nimit Mehta: I think that 2024 is going to be a buckle-down year, but, at the same time, we’ll see a rapid explosion of experimentation. That will definitely stick. This applies to any industry.

Technology 59

Technology Risk Digital Transformation Measurement 59

Domino Data Lab

MAY 8, 2019

In contrast, some researchers who are exploring the definitions, possibilities, and limitations of model interpretation point toward more comprehensive views. There’s no one-size-fits-all definition of what’s required across all verticals. measure the subjects’ ability to trust the models’ results.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

The Unofficial Google Data Science Blog

FEBRUARY 29, 2016

Despite a very large number of experimental units, the experiments conducted by LSOS cannot presume statistical significance of all effects they deem practically significant. The result is that experimenters can’t afford to be sloppy about quantifying uncertainty. In statistics, such segments are often called “blocks” or “strata”.

Experimentation 40

Experimentation Statistics Metrics Measurement 40

Occam's Razor

FEBRUARY 28, 2017

The qualitative surveys measuring unhappiness went down even more than before. You are what you measure. Here are the definitions…. In my definition, this is also online to offline, offline to online or whatever the heck to whatever the heck. Bonus: Remember, you can measure profit everyday in Google Analytics! ].

Key Performance Indicator 107

Key Performance Indicator Metrics Measurement Advertising 107

Domino Data Lab

OCTOBER 2, 2019

As data science work is experimental and probabilistic in nature, data scientists are often faced with making inferences. You’ll measure this effect by looking at a quantity called the average treatment effect (ATE). What you really want to measure is the difference in outcomes. A complementary Domino project is available. .

Machine Learning 68

Machine Learning Measurement Modeling Testing 68

Sisense

JULY 15, 2020

This article will focus on the AI Research (AIR) team’s effort, specifically an experimental combination of Sisense BloX (actionable embedded analytics ) and Quest (an advanced analytics add-on for Sisense) which we called the SEIR app. Dozens of Sisensers took part in project SiCo to create this awesome COVID hub. Envisioning the SEIR app.

Analytics 90

Analytics Modeling Visualization Dashboards 90

Occam's Razor

JANUARY 7, 2014

The organization functions off a clearly defined Digital Marketing & Measurement Model. #1. Before we go too deep, let's get a couple of definitions right first. More on the Digital Marketing & Measurement Model, DMMM, in #2 below.). Four Useless KPI Measurement Techniques. #9: Sad, isn't it?

Data-driven 142

Data-driven Consulting Reporting Marketing 142

Domino Data Lab

FEBRUARY 4, 2019

Spoiler alert: a research field called curiosity-driven learning is emerging at the nexis of experimental cognitive psychology and industry use cases for machine learning, particularly in gaming AI. The ability to measure results (risk-reducing evidence). Ensure a culture that supports a steady process of learning and experimentation.

Data Science 66

Data Science Experimentation Machine Learning Data-driven 66

Occam's Razor

JULY 24, 2018

To name a few: Digital Marketing & Measurement Model | Analytics Ecosystem | Web Analytics 2.0. During a discussion around planning for measurement, a peer was struggling with a unique collection of challenges. You see more digital metrics because digital is more measurable. Especially for the non-obvious problem #2 above.

Analytics 95

Analytics Metrics Strategy Measurement 95

Occam's Razor

MARCH 23, 2010

Too many new things are happening too fast and those of us charged with measuring it have to change the wheels while the bicycle is moving at 30 miles per hour (and this bicycle will become a car before we know it – all while it keeps moving, ever faster). It has simply not had a break to catch a breath and mature. Likely not.

Analytics 119

Analytics Consulting Metrics Experimentation 119