CIO Business Intelligence

AUGUST 15, 2024

Determining the ROI for “ubiquitous” gen AI uses, such as virtual assistants or intelligent chatbots , can be difficult, says Frances Karamouzis, an analyst in the Gartner AI, hyper-automation, and intelligent automation group. CIOs need to be able to articulate the business value and expected ROI of each project.

ROI 143

ROI Cost-Benefit Experimentation Deep Learning 143

datapine

JANUARY 31, 2022

For example, in regards to marketing, traditional advertising methods of spending large amounts of money on TV, radio, and print ads without measuring ROI aren’t working like they used to. They’re about having the mindset of an experimenter and being willing to let data guide a company’s decision-making process. The results?

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

CIO Business Intelligence

MAY 1, 2024

Ready to roll It’s shorter to make a list of organizations that haven’t announced their gen AI investments, pilots, and plans, but relatively few are talking about the specifics of any productivity gains or ROI. Pilots can offer value beyond just experimentation, of course. What are you measuring?

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

datapine

FEBRUARY 4, 2020

Management thinker Peter Drucker once stated, “if you can’t measure it, you can’t improve it” – and he couldn’t be more right. If it has been optimized for SEO though, you shouldn’t stop measuring it after the first week, as it needs a couple of months to reach its “cruising traffic”, and you can get several thousands of monthly visits.

Reporting 280

Reporting Marketing Advertising Metrics 280

DataKitchen

MARCH 8, 2023

This requires a culture of innovation, experimentation, and willingness to take risks and try new approaches. Measuring these goals is very important to success. As the adage goes, You can’t improve what you can’t measure. Does it have ROI? Often the data is part of a complex decision process they are juggling.

Analytics 130

Analytics Manufacturing Data-driven Data Analytics 130

CIO Business Intelligence

AUGUST 14, 2024

Measuring costs and value The other major issue with gen AI is the price. But most licences are for trials, not large scale deployments — usually less than 20% of employees according to Gartner, with early adopters looking at the familiar cost versus ROI equation before expanding. Don’t do it straight across the enterprise.

Enterprise 136

Enterprise Cost-Benefit Experimentation Modeling 136

IBM Big Data Hub

MARCH 4, 2024

By becoming an AI+ enterprise, clients can realize the ROI not only for the AI use case but also for improving the related business and technical capabilities required to deliver AI use cases into production at scale. times higher ROI. times higher ROI. This culture encourages experimentation and expertise growth.

Enterprise 96

Enterprise ROI Risk Strategy 96

CIO Business Intelligence

APRIL 18, 2022

Tech leaders “should have a common language that clearly defines their company’s digital imperatives, with related value measures, that allows the organization to align on strategy across the C-suite and to communicate the strategic value they hope to achieve from it,” Nanda says. They invest in cloud experimentation.

Digital Transformation 145

Digital Transformation Dashboards Cost-Benefit Consulting 145

Occam's Razor

AUGUST 13, 2012

Yehoshua Coren: Best ways to measure user behavior in a multi-touch, multi-device digital world. What's possible to measure. What's not possible to measure. Bjoern Sjut3: My main issue at the moment: How will multi-channel funnels and ROI calculations work in a multi device world? Let's do this!

Modeling 125

Modeling Metrics Data Quality Data-driven 125

Occam's Razor

NOVEMBER 2, 2015

For the rest of this post, I'm going to use the first three to capture the essence of social engagement and brand impact, and one to measure impact on the business. I believe the best way to measure success is to measure the above four metrics (actual interaction/action/outcome). Measure all this Social Media activity.

B2B 167

B2B Metrics Recreation/Entertainment Testing 167

Domino Data Lab

JUNE 23, 2019

Skomoroch proposes that managing ML projects are challenging for organizations because shipping ML projects requires an experimental culture that fundamentally changes how many companies approach building and shipping software. These measurement-obsessed companies have an advantage when it comes to AI.

Management 110

Management Machine Learning Experimentation Metrics 110

IBM Big Data Hub

MAY 17, 2024

Cloud adoption maturity model This maturity model helps measure an organization’s cloud maturity in aggregate. Teams are comfortable with experimentation and skilled in using data to inform business decisions. Service ownership is established and distributed to self-sufficient teams.

Modeling 96

Modeling Cost-Benefit Optimization Digital Transformation 96

Occam's Razor

FEBRUARY 26, 2008

1: Implement a Experimentation & Testing Program. # 1: Implement a Experimentation & Testing Program. Experimentation and Testing: A Primer. Build A Great Web Experimentation & Testing Program. # Often with benchmarks we get into silly arguments like how do they measure this and that etc.

Management 68

Management Experimentation Testing Metrics 68

IBM Big Data Hub

APRIL 18, 2024

While leaders have some reservations about the benefits of current AI, organizations are actively investing in gen AI deployment, significantly increasing budgets, expanding use cases, and transitioning projects from experimentation to production. A key trend is the adoption of multiple models in production.

Cost-Benefit 89

Cost-Benefit Manufacturing Modeling Interactive 89

Ontotext

APRIL 3, 2024

So, this is a big driver for the outcome because when you are saving money for the business, you can measure it and see its value. Nimit Mehta: I think that 2024 is going to be a buckle-down year, but, at the same time, we’ll see a rapid explosion of experimentation. Show me the ROI.” How do I make sure I can manage risk?”

Technology 45

Technology Risk Digital Transformation Measurement 45

CIO Business Intelligence

OCTOBER 1, 2024

Many organizations have struggled to find the ROI after launching AI projects, but there’s a danger in demanding too much too soon, according to IT research and advisory firm Forrester. Measure everything Looking for ROI too soon is often a product of poor planning, says Rowan Curran, an AI and data science analyst at Forrester.

ROI 131

ROI IT Measurement Experimentation 131

Occam's Razor

OCTOBER 19, 2010

If you can show ROI on a DW it would be a good use of your money to go with Omniture Discover, WebTrends Data Mart, Coremetrics Explore. You'll measure Task Completion Rate in 4Q (below). You'll measure Share of Search using Insights for Search (below). Experimentation and Testing Tools [The "Why" – Part 1].

Analytics 136

Analytics Testing Measurement Optimization 136

Occam's Razor

AUGUST 22, 2011

Key To Your Digital Success: Web Analytics Measurement Model. " Measuring Incrementality: Controlled Experiments to the Rescue! Barriers To An Effective Web Measurement Strategy [+ Solutions!]. Measuring Online Engagement: What Role Does Web Analytics Play? "Engagement" How Do I Measure Success?

KPI 125

KPI Metrics Measurement ROI 125

Occam's Razor

MARCH 23, 2010

Too many new things are happening too fast and those of us charged with measuring it have to change the wheels while the bicycle is moving at 30 miles per hour (and this bicycle will become a car before we know it – all while it keeps moving, ever faster). our measurement strategies 2. success measures. Likely not.

Analytics 119

Analytics Consulting Metrics Experimentation 119

Domino Data Lab

MAY 8, 2019

Visualizations are vital in data science work, with the caveat that the information that they convey may be 4-5 layers of abstraction away from the actual business process being measured. measure the subjects’ ability to trust the models’ results. Information can get quite distorted after being abstracted that many times.

Machine Learning 105

Machine Learning Data Science Modeling Visualization 105

Corinium

APRIL 25, 2019

For example, P&C insurance strives to understand its customers and households better through data, to provide better customer service and anticipate insurance needs, as well as accurately measure risks. What do you recommend to organizations to harness this but also show a solid ROI?

Insurance 150

Insurance Risk IoT Data-driven 150

Domino Data Lab

FEBRUARY 4, 2019

Spoiler alert: a research field called curiosity-driven learning is emerging at the nexis of experimental cognitive psychology and industry use cases for machine learning, particularly in gaming AI. The ability to measure results (risk-reducing evidence). Ensure a culture that supports a steady process of learning and experimentation.

Data Science 66

Data Science Experimentation Machine Learning Data-driven 66

Occam's Razor

APRIL 29, 2013

You measure bounce rate and you can find those things, then figure out if the problem is at the source (ads) or destination (your site). Because Likes (and +1s, Followers) measure a fleeting Hello. PS: Bonus : Facebook Advertising / Marketing: Best Metrics, ROI, Business Value. Bad ad creatives. Horrible landing pages.

Marketing 166

Marketing Metrics Advertising Measurement 166

CIO Business Intelligence

OCTOBER 10, 2024

On the one side, Forrester recently warned organizations not to look for AI ROI too soon, because they could miss out on AI’s benefits. Still, a 30% failure rate represents a huge amount of time and money, given how widespread AI experimentation is today. The ROI may be coming from many of these less tangible things,” she says. “The

ROI 135

ROI Experimentation Cost-Benefit Metrics 135

CIO Business Intelligence

FEBRUARY 19, 2025

If 2023 was the year of experimentation with gen AI, 2024 was when companies zeroed in on use cases and started putting pilot projects into production. In a survey of 2,300 IT decision makers that IBM released in December, 47% say theyre already seeing ROI from their AI investments, and 33% say theyre breaking even on AI.

ROI 137

ROI Marketing Sales Software 137

Dataiku

JANUARY 31, 2025

IT leaders, data executives, and business decision-makers are moving beyond early GenAI experimentation and shifting their focus to operationalizing AI at scale for measurable impact. How do we prove ROI and justify continued investment in GenAI?

Experimentation 87

Experimentation ROI Measurement Modeling 87

CIO Business Intelligence

FEBRUARY 5, 2025

Half of CFOs say they plan to cut AI funding if it doesnt show measurable ROI within a year, according to a global survey from accounts payable automation firm Basware, which included 400 CFOs and finance leaders. CIOs are under pressure to validate AI investments and assure CFOs of a clear path of implementation that will ensure ROI.

Cost-Benefit 117

Cost-Benefit ROI Metrics Measurement 117

CIO Business Intelligence

JANUARY 13, 2025

Shift AI experimentation to real-world value Generative AI dominated the headlines in 2024, as organizations launched widespread experiments with the technology to assess its ability to enhance efficiency and deliver new services. Most of all, the following 10 priorities should be at the top of your 2025 to-do list.

Experimentation 140

Experimentation Strategy ROI Digital Transformation 140

CIO Business Intelligence

JANUARY 8, 2025

Measurement of value and focus on short-term ROI could be another deterrent factor for a successful digitalization initiative. Support and encourage experimentation A culture of innovation cannot be built with an attitude of antagonism or aversion towards experimentation.

Experimentation 111

Experimentation Digital Transformation Cost-Benefit ROI 111

CIO Business Intelligence

MARCH 26, 2025

Evaluate ROI and substantiate it with relevance, optimization and impact Utilize your tech investments to deliver financial and operational agility. Collaborate with your CFO Ensure every tech investment drives measurable business outcomes and sustained long-term profits.

Business Objectives 103

Business Objectives Digital Transformation Finance Data-driven 103

CIO Business Intelligence

DECEMBER 23, 2024

AI investment and pressure grew upward As AI has moved from emerging to mainstream, and organizations matured in their ability to harness AIs potential over the past year or two, CEOs now expect less experimentation and more AI projects that deliver outcomes with measurable business value.

IT 134

IT Strategy Technology Experimentation 134

Timo Elliott

JANUARY 27, 2025

Cultural Resistance : Adopting AI often requires a shift in corporate culture, encouraging experimentation and data-driven decision-making. Uncertain ROI : Many companies struggle to quantify the immediate benefits of AI, making it difficult to justify the initial investment.

Manufacturing 52

Manufacturing Cost-Benefit Experimentation ROI 52

O'Reilly on Data

APRIL 15, 2025

Heres a common scene from my consulting work: AI TEAM Heres our agent architectureweve got RAG here, a router there, and were using this new framework for ME [Holding up my hand to pause the enthusiastic tech lead] Can you show me how youre measuring if any of this actually works? Instead, they obsess over measurement and iteration.

Experimentation 299

Experimentation Testing Metrics Measurement 299