Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers.

Experimentation 103

Experimentation Testing Optimization Measurement 103

O'Reilly on Data

MARCH 31, 2020

You’re responsible for the design, the product-market fit, and ultimately for getting the product out the door. This has serious implications for software testing, versioning, deployment, and other core development processes. The ability to make decisions based on data analytics is a prerequisite for an “experimental culture.”

Management 339

Management Machine Learning Experimentation Metrics 339

CIO Business Intelligence

FEBRUARY 21, 2024

If you don’t know these tools, you’re at a disadvantage in the job market.” We just happen to have some of the best athletic programs in the country as well, requiring us to stay on the cutting edge of sports marketing and entertainment.” We’ve done a lot of experimentation on these adaptive tools that use AI,” says Ventimiglia.

Recreation/Entertainment 135

Recreation/Entertainment Experimentation Data-driven Modeling 135

Occam's Razor

MARCH 23, 2021

This is very hard to do, we now have a proven seven-step experimentation process, with one of the coolest algorithms to pick matched-markets (normally the kiss of death of any large-scale geo experiment). The first component is a gloriously scaled global creative pre-testing program. Matched market tests.

Analytics 133

Analytics Scorecard Strategy Testing 133

CIO Business Intelligence

MAY 16, 2023

It surpasses blockchain and metaverse projects, which are viewed as experimental or in the pilot stage, especially by established enterprises. Gaming: Companies behind popular video games argue that their virtual worlds are the closest thing resembling a “metaverse” on the market.

Advertising 105

Advertising Recreation/Entertainment Experimentation Enterprise 105

Occam's Razor

AUGUST 22, 2011

To assist with that process everything's organized into these sections: ~ Digital Marketing: "What is amazing out there? " ~ Web Metrics: "What is a KPI? " + Standard Metrics Revisited Series. Digital Marketing: "What is amazing out there? Online Marketing Still A Faith Based Initiative.

KPI 125

KPI Metrics Measurement ROI 125

Sisense

JANUARY 21, 2020

One of the simplest ways to start exploring your data is to aggregate the metrics you are interested in by their relevant dimensions. You’re working for an ice cream company that is about to market a brand new special-edition flavor. A new drug promising to reduce the risk of heart attack was tested with two groups.

Testing 104

Testing Risk Data-driven Statistics 104

CIO Business Intelligence

JUNE 27, 2022

Tyson: Although there are other tools in the market for application monitoring, Sentry seems to be the leader. Tyson: You’re coming up to two years since you added Sentry’s ability to monitor for performance which has some pretty fine-grained metrics in terms of identifying where bottlenecks are located in code. I thought, really?!

Software 115

Software Experimentation Testing Marketing 115

datapine

FEBRUARY 4, 2020

Let’s face it: every serious business that wants to generate leads and revenue needs to have a marketing strategy that will help them in their quest for profit. Be it in marketing, or in sales, finance or for executives, reports are essential to assess your activity and evaluate the results. What Is A Marketing Report?

Reporting 280

Reporting Marketing Advertising Metrics 280

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times. Now nearly half of code suggestions are accepted.

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

Occam's Razor

JULY 6, 2016

Yes, I worry that Analysts, and Marketers, are spending too much time with their head buried in custom reports and advance segments and smart calculated metrics and strategic or tactical dashboards. Lab usability testing and online surveys both provide great strategies to obsess about user centric design. United Breaks Hearts.

Testing 87

Testing Marketing Experimentation Cost-Benefit 87

Occam's Razor

OCTOBER 21, 2013

This led to the problem we, Marketers, SEOs, Analysts, fondly refer to as not provided. The reason I love the above view is that for each channel, I'm able to present our management team a simple, yet powerful, understanding of the contribution of our marketing channels – including search. Controlled experimentation.

Optimization 156

Optimization Reporting Experimentation Metrics 156

Insight

MARCH 12, 2020

I tested several different flavors of BERT for use as synopsis classifiers before settling on the DistilBERT model from Hugging Face. After some experimentation, I landed on a strategy I’ll call ‘warm encoding’: if greater than 1% of tags were in a particular class, I encoded the book as belonging to that class, non-exclusively.

Modeling 59

Modeling Metadata Publishing Sales 59

Occam's Razor

JULY 18, 2011

To not have it as an active part of your marketing portfolio is sub-optimal. The only requirement is that your mental model (and indeed, company culture) should be solidly rooted in permission marketing. Embrace permission marketing and email will be a surprising and loyal BFF. Optimal Acquisition Email Metrics.

Metrics 138

Metrics Marketing Measurement Cost-Benefit 138

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

O'Reilly on Data

OCTOBER 13, 2020

Our previous articles in this series introduce our own take on AI product management , discuss the skills that AI product managers need , and detail how to bring an AI product to market. In Bringing an AI Product to Market , we distinguished the debugging phase of product development from pre-deployment evaluation and testing.

Management 251

Management Metrics Machine Learning Modeling 251

Occam's Razor

DECEMBER 9, 2009

As such, we are not a part of the central Marketing (and Marketing Intelligence) teams at corporate. The four pronged real world tested probing and loaded with politics framework to find a home for Web Analytics: 1. The Analysts / Marketers / IT dudes keep churning data and sending the insights but nothing every changes.

Analytics 95

Analytics Data-driven Marketing Sales 95

Occam's Razor

NOVEMBER 13, 2009

It is not just a business book, though every dna strand in this book is more about online marketing than online analytics. Chapter 3 The Awesome World of Clickstream Analysis: Metrics. Chapter 7 Failing Faster: Unleashing the Power of Testing and Experimentation. Sure you've heard of A/B and multivariate testing.

Analytics 71

Analytics Metrics Measurement Testing 71

DataKitchen

APRIL 13, 2021

As a result, vendors that market DataOps capabilities have grown in pace with the popularity of the practice. Testing and Data Observability. Please let us know if we have forgotten anyone or if you have any comments (marketing@datakitchen.io). Testing and Data Observability. Meta-Orchestration. Continuous Deployment.

Testing 300

Testing Machine Learning Consulting Data Quality 300

Occam's Razor

FEBRUARY 26, 2008

" Or " I proposed testing / surveys / competitive intelligence / Analysts but I was shot down." 1: Implement a Experimentation & Testing Program. # 1: Implement a Experimentation & Testing Program. Here is data from our latest test." And now you have no excuse to avoid testing.

Management 68

Management Experimentation Testing Metrics 68

Occam's Razor

SEPTEMBER 15, 2014

If you look at the mobile marketing strategies, you will see they don't reflect this shift to mobile. They will need two different implementations, it is quite likely that you will end up with two sets of metrics (more people focused for mobile apps, more visit focused for sites). Media-Mix Modeling/Experimentation.

Metrics 143

Metrics Reporting Analytics Marketing 143

datapine

MAY 20, 2020

Do you want to make your content marketing campaigns more focused or streamline your customer service expenditure? To inspire your customer relationship management report for managing your metrics, explore our cutting-edge selection of KPI examples. Test, tweak, evolve. Do you want to build social media engagement?

Dashboards 265

Dashboards Reporting KPI Visualization 265

CIO Business Intelligence

MARCH 5, 2024

Many other platforms, such as Coveo’s Relative Generative Answering , Quickbase AI , and LaunchDarkly’s Product Experimentation , have embedded virtual assistant capabilities but don’t brand them copilots. Today, top AI-assistant capabilities delivering results include generating code, test cases, and documentation.

Cost-Benefit 124

Cost-Benefit Advertising Experimentation Software 124

Occam's Razor

JUNE 6, 2007

Bonus: Interactive CD: Contains six podcasts, one video, two web analytics metrics definitions documents and five insightful powerpoint presentations. Experimentation & Testing (A/B, Multivariate, you name it). What ideas to test first on your site? Immediately actionable web analytics (your biggest worries covered).

Analytics 60

Analytics Metrics Marketing Dashboards 60

Sisense

MAY 18, 2021

If marketing were an apple pie, data would be the apples — without data supporting your marketing program, it might look good from the outside, but inside it’s hollow. In a recent survey from Villanova University, 100% of marketers said data analytics has an essential role in marketing’s future.

Marketing 52

Marketing Data Analytics Analytics Metrics 52

Occam's Razor

NOVEMBER 12, 2006

The problem is that you are there just to look at the car, maybe take it for a test drive. Using Market Research or Website Surveys or other methods attempt to compute why Visitors come to your website (I call this Visitor Primary Purpose). Focus on the Why (use Surveys or Lab Usability or Experimentation & Testing for example).

Measurement 99

Measurement Analytics Optimization Marketing 99

DataKitchen

JANUARY 17, 2022

Customers and market forces drive deadlines and timeframes for analytics deliverables regardless of the level of effort required. Testing and validating analytics took as long or longer than creating the analytics. This large enterprise has many products and brands with overlapping marketing campaigns. Data is not static.

Experimentation 130

Experimentation Data Lake Marketing Predictive Modeling 130

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. Yet, this challenge is not insurmountable. for what is and isn’t possible) to address these challenges. Transcript.

Management 110

Management Machine Learning Experimentation Metrics 110

CIO Business Intelligence

APRIL 18, 2022

They also advise communicating the dashboard’s value consistently since that will drive effective dashboard use, both to increase adoption and to improve company performance on key dashboard metrics, the brief says. Our internal QA team now focuses 100% on automated testing and managing the queue from the crowdsourced operation.

Digital Transformation 145

Digital Transformation Dashboards Cost-Benefit Consulting 145

Occam's Razor

NOVEMBER 17, 2008

There is a tendency to think experimentation and testing is optional. Just don't fall for their bashing of all other vendors or their silly claims, false, of "superiority" in terms of running 19 billion combinations of tests or the bonus feature of helping you into your underwear each morning. And I meant every word of it.

Testing 113

Testing Experimentation Risk Optimization 113

Occam's Razor

JANUARY 9, 2012

The real problem is that our management teams lack imagination when it comes to the web, and our marketing executives continue to do TV on Twitter, catalogs on display ads, irrelevant shouting on search, etc. The problem is Marketing and lack of imagination in using the web/digital channels. The problem, it turns out, is not data.

Marketing 142

Marketing B2B Measurement Risk 142

The Unofficial Google Data Science Blog

APRIL 16, 2019

by MICHAEL FORTE Large-scale live experimentation is a big part of online product development. This means a small and growing product has to use experimentation differently and very carefully. This blog post is about experimentation in this regime. Such decisions involve an actual hypothesis test on specific metrics (e.g.

Experimentation 68

Experimentation Sales Metrics Measurement 68

The Unofficial Google Data Science Blog

JUNE 30, 2016

In an ideal world, experimentation through randomization of the treatment assignment allows the identification and consistent estimation of causal effects. A naïve way to solve this problem would be to compare the proportion of buyers between the exposed and unexposed groups, using a simple test for equality of means.

Statistics 44

Statistics Optimization Modeling Experimentation 44

Occam's Razor

AUGUST 12, 2013

MCA-O2S covers the challenge of attributing the offline impact (revenue/brand value/butts in seats/phone calls/etc) driven by online marketing and advertising. MCA-AMS covers the challenge of attributing accurate impact of our marketing and advertising efforts across multiple devices (desktop, laptop, mobile, TV). It is sweet.

Modeling 162

Modeling Optimization Marketing Interactive 162

DataRobot Blog

FEBRUARY 15, 2023

In such a hot market, how can teams leverage AI to ensure that they are assessing the right values in their respective markets? The real estate market changes over time, so it’s important that our model learns from past data and is tested on a time frame from the future.

Modeling 52

Modeling Machine Learning Cost-Benefit Metrics 52

The Unofficial Google Data Science Blog

MAY 31, 2016

Similarly, we could test the effectiveness of a search ad compared to showing only organic search results. A geo experiment is an experiment where the experimental units are defined by geographic regions. Such regions are often referred to as Generalized Market Areas (GMAs) or simply geos. by turning campaigns off).

Advertising 42

Advertising Testing Sales Statistics 42

Occam's Razor

MARCH 23, 2010

Because every tool uses its own sweet metrics definitions, cookie rules, session start and end rules and so much more. In case of WebTrends replace those terms with Marketing Intelligence / Visitor Data Mart etc. Yet case studies in some sense reduced risk, even if they were simply over blown marketing fluff written by the vendor.

Analytics 119

Analytics Consulting Metrics Experimentation 119