Occam's Razor

MAY 22, 2006

This post is a primer on the delightful world of testing and experimentation (A/B, Multivariate, and a new term from me: Experience Testing). Experimentation and testing help us figure out we are wrong, quickly and repeatedly and if you think about it that is a great thing for our customers, and for our employers.

Experimentation 103

Experimentation Testing Optimization Measurement 103

Rocket-Powered Data Science

FEBRUARY 15, 2023

3) How do we get started, when, who will be involved, and what are the targeted benefits, results, outcomes, and consequences (including risks)? encouraging and rewarding) a culture of experimentation across the organization. It is important to realize that the usual “hype cycle” rules prevail in such cases as this.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

CIO Business Intelligence

APRIL 25, 2024

Fits and starts As most CIOs have experienced, embracing emerging technologies comes with its share of experimentation and setbacks. Here, it was believed an LLM would help, as an oft-touted benefit of LLMs is their speed, enabling them to complete complex steps rapidly. The initial deliverables “felt lacking,” Bottaro said.

IT 139

IT Cost-Benefit Experimentation Metrics 139

AWS Big Data

NOVEMBER 19, 2024

This offering is designed to provide an even more cost-effective solution for running Airflow environments in the cloud. micro characteristics, key benefits, ideal use cases, and how you can set up an Amazon MWAA environment based on this new environment class. micro reflect a balance between functionality and cost-effectiveness.

Metadata 110

Metadata Cost-Benefit Metrics Optimization 110

CIO Business Intelligence

MAY 21, 2024

Along with code-generating copilots and text-to-image generators, which leverage a combination of LLMs and diffusion processing, LLMs are at the core of most generative AI experimentation in business today. And the benefits of MakeShift’s use of AI are beginning to multiply. It’s fabulous.”

IT 131

IT Cost-Benefit Experimentation Forecasting 131

CIO Business Intelligence

JANUARY 2, 2024

What benefit does AI serve to that department? Bring the whole organization on the AI journey CIOs also see the need to bring everyone along on that AI journey, something that takes a well-articulated narrative about the benefits AI can bring to those who are and will be impacted by the technology. We’re piloting, PoC-ing.

IT 144

IT Cost-Benefit Statistics Consulting 144

AWS Big Data

JANUARY 15, 2025

EUROGATE is a leading independent container terminal operator in Europe, known for its reliable and professional container handling services. Every day, EUROGATE handles thousands of freight containers moving in and out of ports as part of global supply chains. From here, the metadata is published to Amazon DataZone by using AWS Glue Data Catalog.

IoT 107

IoT Machine Learning Metadata Data-driven 107

CIO Business Intelligence

JULY 18, 2024

The early bills for generative AI experimentation are coming in, and many CIOs are finding them more hefty than they’d like — some with only themselves to blame. By understanding their options and leveraging GPU-as-a-service, CIOs can optimize genAI hardware costs and maintain processing power for innovation.”

Cost-Benefit 136

Cost-Benefit Experimentation Consulting Modeling 136

CIO Business Intelligence

MARCH 5, 2025

Two years of experimentation may have given rise to several valuable use cases for gen AI , but during the same period, IT leaders have also learned that the new, fast-evolving technology isnt something to jump into blindly. Before considering a project, Lexmark first makes sure the problem is worth tackling.

Testing 99

Testing Measurement Technology Experimentation 99

CIO Business Intelligence

MAY 23, 2022

Enterprises moving their artificial intelligence projects into full scale development are discovering escalating costs based on initial infrastructure choices. Many companies whose AI model training infrastructure is not proximal to their data lake incur steeper costs as the data sets grow larger and AI models become more complex.

Experimentation 137

Experimentation Cost-Benefit Data Lake Data Science 137

Corinium

JUNE 6, 2019

For instance, for a variety of reasons, in the short term, CDAOS are challenged with quantifying the benefits of analytics’ investments. Also, design thinking should play a large role in analytics in terms of how it will benefit the organization and exactly how people will react to and adopt the resulting insights. It is fast and slow.

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

CIO Business Intelligence

SEPTEMBER 10, 2024

Our IT evolution Having worked primarily in traditionally structured industries like oil and gas, government, education and finance, I’ve witnessed firsthand how technology was once considered a commodity, a cost center. My experience aligns with this trend. However, two crucial misconceptions persist. IT’s image problem?

IT 122

IT Risk Risk Management Predictive Modeling 122

CIO Business Intelligence

NOVEMBER 21, 2023

Ask IT leaders about their challenges with shadow IT, and most will cite the kinds of security, operational, and integration risks that give shadow IT its bad rep. That’s not to downplay the inherent risks of shadow IT. There are ample reasons why 77% of IT professionals are concerned about shadow IT, according to a report from Entrust.

IT 137

IT Risk Cost-Benefit Data Science 137

CIO Business Intelligence

MARCH 6, 2023

CIOs along with researchers, consultants, and advisors agree that IT must change itself, how it works and how it organizes its workers, if it wants to gain the most benefits out of cloud computing. As CIO Neil Holden moved his company, Halfords Group, further into the cloud, he sought to do more than simply “lift-and-shift” IT operations.

IT 140

IT Strategy Cost-Benefit Consulting 140

CIO Business Intelligence

MARCH 19, 2025

This enforces the need for good data governance, as AI models will surface incorrect data more frequently, and most likely at a greater cost to the business. CIOs have the daunting task of educating it on the various flavors of this capability, and steering them to the most beneficial investments and strategies.

Metrics 64

Metrics Enterprise Cost-Benefit Experimentation 64

CIO Business Intelligence

AUGUST 14, 2024

The cost of OpenAI is the same whether you buy it directly or through Azure. Beyond the ubiquity of ChatGPT, CIOs will find obvious advantages working with a familiar enterprise supplier that understands their needs better than many AI startups, and promises integrations with existing enterprise tools.

Enterprise 136

Enterprise Cost-Benefit Experimentation Modeling 136

CIO Business Intelligence

FEBRUARY 22, 2024

With the aim to accelerate innovation and transform its digital infrastructures and services, Ferrovial created its Digital Hub to serve as a meeting point where research and experimentation with digital strategies could, for example, provide new sources of income and improve company operations.

IT 105

IT Digital Transformation IoT Cost-Benefit 105

CIO Business Intelligence

JULY 5, 2024

So, to maximize the ROI of gen AI efforts and investments, it’s important to move from ad-hoc experimentation to a more purposeful strategy and systematic approach to implementation. Here are five best practices to get the most business benefit from gen AI. What makes generative AI implementations so challenging?

ROI 124

ROI Experimentation Risk Cost-Benefit 124

CIO Business Intelligence

OCTOBER 9, 2022

Currently, equipment costs are high while comfort is low (even the most hardened gamers need a break after an hour or so fully kitted up), and accessible computing power is nowhere near where it will need to be. Feasibility is perhaps the simplest of these three lenses, as it’s fairly consistent across business types and sizes.

Cost-Benefit 142

Cost-Benefit Manufacturing Experimentation Risk 142

datapine

FEBRUARY 4, 2020

Let’s face it: every serious business that wants to generate leads and revenue needs to have a marketing strategy that will help them in their quest for profit. Ultimately, it will provide a clear insight into relevant KPIs and build a solid foundation for increasing conversions. How do you know that? Or drastically change for another path?

Reporting 280

Reporting Marketing Advertising Metrics 280

CIO Business Intelligence

MAY 1, 2024

Pilots can offer value beyond just experimentation, of course. McKinsey reports that industrial design teams using LLM-powered summaries of user research and AI-generated images for ideation and experimentation sometimes see a reduction upward of 70% in product development cycle times.

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

Smart Data Collective

OCTOBER 13, 2021

AI technology moves innovation forward by boosting tinkering and experimentation, accelerating the innovation process. One of the biggest benefits of AI is that it has led to new breakthroughs in automation. One of the best benefits of AI is that it can help improve the user experience through features like personalization.

Technology 133

Technology Cost-Benefit Experimentation Machine Learning 133

CIO Business Intelligence

SEPTEMBER 9, 2024

But there comes a point in a new technology when its potential benefits become clear even if the exact shape of its evolution is opaque. Earlier this year, consulting firm BCG published a survey of 1,400 C-suite executives and more than half expected AI and gen AI to deliver cost savings this year.

Cost-Benefit 131

Cost-Benefit Experimentation Digital Transformation ROI 131

CIO Business Intelligence

AUGUST 15, 2024

Many of those gen AI projects will fail because of poor data quality, inadequate risk controls, unclear business value , or escalating costs , Gartner predicts. Gen AI projects can cost millions of dollars to implement and incur huge ongoing costs, Gartner notes. For example, a gen AI virtual assistant can cost $5 million to $6.5

ROI 143

ROI Cost-Benefit Experimentation Deep Learning 143

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. Davé and his team’s achievements in AI are due in large part to creating opportunities for experimentation — and ensuring those experiments align with CBRE’s business strategy. And those experiments have paid off. Human intervention is still necessary.

Experimentation 130

Experimentation Strategy Machine Learning Interactive 130

Smart Data Collective

OCTOBER 13, 2022

These patterns could then be used as the basis for additional experimentation by scientists or engineers. The technique is helping product design firm Seattle reduce costs and improve the quality of its products. Though AI has many benefits in product R&D, it has some limitations in application. Generative Design.

Testing 128

Testing Cost-Benefit Experimentation Optimization 128

CIO Business Intelligence

OCTOBER 3, 2024

It’s a natural fit and will be interesting to see how these ensemble AI models work and what use cases will go from experimentation to production,” says Dyer. A combination of mainframe and cloud for different tasks might be a more flexible, cost-effective solution.” At least IBM believes so. It’s not the technologies they use.

Machine Learning 140

Machine Learning Experimentation Insurance Modeling 140

CIO Business Intelligence

OCTOBER 3, 2024

Medium companies Medium-sized companies—501 to 5,000 employees—were characterized by agility and a strong focus on GenAI experimentation. Resource constraints and the need for immediate, tangible benefits are likely to have shaped their more cautious, results-focused approach. They also have the means to back it up.

Testing 129

Testing Cost-Benefit Experimentation Risk 129

CIO Business Intelligence

JULY 24, 2024

Key strategies for exploration: Experimentation: Conduct small-scale experiments. Key strategies for evolution: Maintain flexible architecture: Maintain the modularity and scalability of solutions to enable cost-effective capability expansions as requirements evolve. Regularly evaluate performance to prioritize enhancements.

Technology 98

Technology Experimentation Cost-Benefit Digital Transformation 98

CIO Business Intelligence

SEPTEMBER 12, 2024

If the code isn’t appropriately tested and validated, the software in which it’s embedded may be unstable or error-prone, presenting long-term maintenance issues and costs. A routine audit uncovers severe compliance issues with how the tool accesses and stores data.

Experimentation 111

Experimentation Risk Cost-Benefit Strategy 111

Cloudera

JANUARY 28, 2022

Customers vary widely on the topic of public cloud – what data sources, what use cases are right for public cloud deployments – beyond sandbox, experimentation efforts. Private cloud continues to gain traction with firms realizing the benefits of greater flexibility and dynamic scalability. Cost Management.

Slice and Dice 115

Slice and Dice Cost-Benefit Data Governance Risk 115

CIO Business Intelligence

OCTOBER 15, 2024

It may surprise you, but DevOps has been around for nearly two decades. Driven by the development community’s desire for more capabilities and controls when deploying applications, DevOps gained momentum in 2011 in the enterprise with a positive outlook from Gartner and in 2015 when the Scaled Agile Framework (SAFe) incorporated DevOps.

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

APRIL 24, 2024

The first use of generative AI in companies tends to be for productivity improvements and cost cutting. But there are only so many costs that can be cut. CIOs are well positioned to cut costs since they’re usually well acquainted with a company’s digital processes, having helped set them up in the first place.

Sales 128

Sales Finance Experimentation Cost-Benefit 128

CIO Business Intelligence

MAY 23, 2022

For many nascent AI projects in the prototyping and experimentation phase, the cloud works just fine. But companies often discover that as data sets grow in volume and AI model complexity increases, the escalating cost of compute cycles, data movement, and storage can spiral out of control. Cloud Architecture, IT Leadership

Cost-Benefit 98

Cost-Benefit Experimentation Data Lake Deep Learning 98

CIO Business Intelligence

FEBRUARY 16, 2024

For most organizations, a shift to the cloud brings scalability, access to innovative tools, and the possibility of cost savings. When you’re introducing many new applications, the ease of getting them up and running and lowered costs [on the cloud] is tremendously beneficial,” he says. An early partner of Amazon, the Roseburg, N.J.-based

Digital Transformation 116

Digital Transformation Experimentation Cost-Benefit Machine Learning 116

CIO Business Intelligence

OCTOBER 4, 2023

One of the first things they built was an HR chatbot, which provided benefits recommendations that unnecessarily exposed them to massive liability. For example, if the HR tool recommended the wrong option, an employee could miss the benefits window for an entire year. But it still creates unappetizing combinations.

Cost-Benefit 141

Cost-Benefit Manufacturing Machine Learning Experimentation 141

CIO Business Intelligence

MARCH 12, 2024

But early returns indicate the technology can provide benefits for the process of creating and enhancing applications, with caveats. Software and coding development remain a high-value area for experimentation, in addition to content development and knowledge management, in an effort to boost operational efficiencies,” he says.

Software 131

Software Experimentation Testing Cost-Benefit 131