CIO Business Intelligence

NOVEMBER 27, 2024

People : To implement a successful Operational AI strategy, an organization needs a dedicated ML platform team to manage the tools and processes required to operationalize AI models. Adopting Operational AI Organizations looking to adopt Operational AI must consider three core implementation pillars: people, process, and technology.

Data Architecture 128

Data Architecture Cost-Benefit Machine Learning Experimentation 128

CIO Business Intelligence

DECEMBER 17, 2024

While genAI has been a hot topic for the past couple of years, organizations have largely focused on experimentation. What are the associated risks and costs, including operational, reputational, and competitive? Increase adoption through change management. In 2025, thats going to change.

Experimentation 98

Experimentation ROI Risk Data Quality 98

CIO Business Intelligence

NOVEMBER 11, 2024

Nate Melby, CIO of Dairyland Power Cooperative, says the Midwestern utility has been churning out large language models (LLMs) that not only automate document summarization but also help manage power grids during storms, for example. The thing about the AI stuff is it’s really cheap, if you do it right,” Beswick says.

ROI 137

ROI Cost-Benefit Risk Experimentation 137

CIO Business Intelligence

JANUARY 6, 2025

One of them is Katherine Wetmur, CIO for cyber, data, risk, and resilience at Morgan Stanley. Wetmur says Morgan Stanley has been using modern data science, AI, and machine learning for years to analyze data and activity, pinpoint risks, and initiate mitigation, noting that teams at the firm have earned patents in this space.

IT 140

IT Cost-Benefit Measurement Experimentation 140

CIO Business Intelligence

JANUARY 21, 2025

CIOs feeling the pressure will likely seek more pragmatic AI applications, platform simplifications, and risk management practices that have short-term benefits while becoming force multipliers to longer-term financial returns. CIOs should consider placing these five AI bets in 2025.

Cost-Benefit 142

Cost-Benefit Data-driven Strategy Marketing 142

O'Reilly on Data

MARCH 18, 2020

Whether it’s controlling for common risk factors—bias in model development, missing or poorly conditioned data, the tendency of models to degrade in production—or instantiating formal processes to promote data governance, adopters will have their work cut out for them as they work to establish reliable AI production lines. But what kind?

Enterprise 349

Enterprise Deep Learning Data Governance Risk 349

CIO Business Intelligence

OCTOBER 24, 2024

As they look to operationalize lessons learned through experimentation, they will deliver short-term wins and successfully play the gen AI — and other emerging tech — long game,” Leaver said. Determining the optimal level of autonomy to balance risk and efficiency will challenge business leaders,” Le Clair said.

ROI 127

ROI Data-driven Enterprise Experimentation 127

CIO Business Intelligence

AUGUST 21, 2024

Adding smarter AI also adds risk, of course. “At The big risk is you take the humans out of the loop when you let these into the wild.” When it comes to security, though, agentic AI is a double-edged sword with too many risks to count, he says. “We That means the projects are evaluated for the amount of risk they involve.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

OCTOBER 17, 2024

While tech debt refers to shortcuts taken in implementation that need to be addressed later, digital addiction results in the accumulation of poorly vetted, misused, or unnecessary technologies that generate costs and risks. Striking this balance is delicate and requires careful management. Assume unknown unknowns.

Risk 108

Risk Experimentation Testing Strategy 108

O'Reilly on Data

MARCH 25, 2025

With the advent of generative AI, therell be significant opportunities for product managers, designers, executives, and more traditional software engineers to contribute to and build AI-powered software. Hallucination risk : Add stronger grounding in retrieval or prompt modifications.

Testing 174

Testing Data-driven Software Measurement 174

Dataiku

OCTOBER 2, 2019

It’s probably safe to say that for at least some of those explorers, the prospect of risk when it comes to data and AI projects is paralyzing, causing them to stay in a phase of experimentation.

Risk 14

Risk Experimentation Management Machine Learning 14

Domino Data Lab

MAY 15, 2019

It focuses on his ML product management insights and lessons learned. If you are interested in hearing more practical insights on ML or AI product management, then consider attending Pete’s upcoming session at Rev. I was fortunate to see an early iteration of Pete Skomoroch ’s ML product management presentation in November 2018.

Machine Learning 111

Machine Learning Management Experimentation Data Science 111

Rocket-Powered Data Science

FEBRUARY 15, 2023

3) How do we get started, when, who will be involved, and what are the targeted benefits, results, outcomes, and consequences (including risks)? Friction occurs when there is resistance to change or to success somewhere in the project lifecycle or management chain. Test early and often. Expect continuous improvement.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

CIO Business Intelligence

JANUARY 14, 2025

In todays digital economy, business objectives like becoming a leading global wealth management firm or being a premier destination for top talent demand more than just technical excellence. Most importantly, architects make difficult problems manageable. The stakes have never been higher. Shawn McCarthy 3.

Enterprise 119

Enterprise Technology Metrics Measurement 119

CIO Business Intelligence

SEPTEMBER 23, 2024

From the rise of value-based payment models to the upheaval caused by the pandemic to the transformation of technology used in everything from risk stratification to payment integrity, radical change has been the only constant for health plans. The culprit keeping these aspirations in check? It is still the data.

Management 115

Management Experimentation Data-driven Strategy 115

CIO Business Intelligence

JUNE 15, 2023

Despite headlines warning that artificial intelligence poses a profound risk to society , workers are curious, optimistic, and confident about the arrival of AI in the enterprise, and becoming more so with time, according to a recent survey by Boston Consulting Group (BCG). For many, their feelings are based on sound experience.

Management 116

Management Experimentation Optimization Enterprise 116

CIO Business Intelligence

MARCH 5, 2025

Two years of experimentation may have given rise to several valuable use cases for gen AI , but during the same period, IT leaders have also learned that the new, fast-evolving technology isnt something to jump into blindly. Use a mix of established and promising small players To mitigate risk, Gupta rarely uses small vendors on big projects.

Testing 99

Testing Measurement Technology Experimentation 99

Aryng

APRIL 22, 2022

Most managers are good at formulating innovative […] The post How to differentiate the thin line separating innovation and risk in experimentation appeared first on Aryng's Blog. We have seen this as a general trend in start-ups, and we know that it’s an awful feeling!

Experimentation 52

Experimentation Risk Management IT 52

CIO Business Intelligence

SEPTEMBER 12, 2024

We spoke with several IT leaders for their insights on what might make an IT worker safe or vulnerable in this environment and what steps CIOs can take to build and manage an IT team for survival. Gray: “IT employees who do not embrace AI will put their jobs at risk. That something else is where the value is.

Management 115

Management IT Experimentation Technology 115

DataRobot Blog

DECEMBER 6, 2022

Other organizations are just discovering how to apply AI to accelerate experimentation time frames and find the best models to produce results. Taking a Multi-Tiered Approach to Model Risk Management. Data scientists are in demand: the U.S. Explore these 10 popular blogs that help data scientists drive better data decisions.

Experimentation 145

Experimentation Forecasting Data-driven Machine Learning 145

CIO Business Intelligence

AUGUST 20, 2024

High performance back then generally focused on delivery — a contrast to previous generations of IT where business and IT alignment was an issue, and teams struggled to deliver with waterfall project management practices. What is a high-performance team today?

IT 141

IT Digital Transformation Experimentation Risk 141

CIO Business Intelligence

NOVEMBER 7, 2024

This team addresses potential risks, manages AI across the company, provides guidance, implements necessary training, and keeps abreast of emerging regulatory changes. This initiative offers a safe environment for learning and experimentation. Fast-forward to today, about 18 months into our journey, and we’re at phase three.

Insurance 120

Insurance Experimentation Testing Technology 120

CIO Business Intelligence

JANUARY 10, 2025

Enterprise technology providers will introduce agentic AI capabilities throughout 2025, enabling organizations to move from experimentation and piloting to broad-scale deployment and integration into existing workstreams, said Todd Lohr, Head of Ecosystems at KPMGs US Advisory division. However, only 12% have deployed such tools to date.

ROI 81

ROI Enterprise Broadcasting Metrics 81

Domino Data Lab

JUNE 23, 2019

Pete Skomoroch presented “ Product Management for AI ” at Rev. Pete Skomoroch ’s “ Product Management for AI ”session at Rev provided a “crash course” on what product managers and leaders need to know about shipping machine learning (ML) projects and how to navigate key challenges. Session Summary. It is similar to R&D.

Management 110

Management Machine Learning Experimentation Metrics 110

CIO Business Intelligence

JULY 5, 2024

So, to maximize the ROI of gen AI efforts and investments, it’s important to move from ad-hoc experimentation to a more purposeful strategy and systematic approach to implementation. Set your holistic gen AI strategy Defining a gen AI strategy should connect into a broader approach to AI, automation, and data management.

ROI 124

ROI Experimentation Risk Cost-Benefit 124

Domino Data Lab

AUGUST 6, 2021

Model Risk Management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

CIO Business Intelligence

JULY 3, 2023

For the last 30 years, the dream of being able to collect, manage and make use of the collected knowledge assets of an organization has never been truly realized. But the rise of large language models (LLMs) is starting to make true knowledge management (KM) a reality. The knowledge management dream is becoming a reality.

Management 105

Management Experimentation Modeling Enterprise 105

CIO Business Intelligence

AUGUST 16, 2023

Today, state-owned Svevia is the country’s largest company in the operation and maintenance of roads and bridges, and manages over 50% of the road network yet, just like in the construction industry, it’s been relatively late to digitization. CIO, Cloud Management, Data Management, Digital Transformation, IT Leadership

Risk 98

Risk Cost-Benefit Internet of Things Digital Transformation 98

CIO Business Intelligence

OCTOBER 15, 2024

Recommendation : Ask leaders for their understanding of key practices such as agile, DevOps, and product management, and differences in core principles, methodologies, and tools will surface. Shortchanging end-user and developer experiences Many DevOps practices focus on automation, such as CI/CD and infrastructure as code.

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

SEPTEMBER 10, 2024

As we navigate this terrain, it’s essential to consider the potential risks and compliance challenges alongside the opportunities for innovation. Decision-making should be deliberate and strategic, rather than purely reactive to technological advancements. Embracing the technology while carefully managing its integration is crucial.

IT 122

IT Risk Risk Management Predictive Modeling 122

CIO Business Intelligence

SEPTEMBER 12, 2024

A product manager is under immense pressure to deliver complex customer insights that could pivot the company’s product strategy. His manager praises his efficiency and the depth and breadth of insights he produces. Imagine a highly competitive market where the urgency to innovate is high.

Experimentation 111

Experimentation Risk Cost-Benefit Strategy 111

CIO Business Intelligence

FEBRUARY 9, 2024

Establish a corporate use policy As I mentioned in an earlier article , a corporate use policy and associated training can help educate employees on some risks and pitfalls of the technology, and provide rules and recommendations to get the most out of the tech, and, therefore, the most business value without putting the organization at risk.

IT 138

IT Experimentation Risk Testing 138

Corinium

JUNE 6, 2019

Regulations and compliance requirements, especially around pricing, risk selection, etc., Moreover, rapid and full adoption of analytics insights can hit speed bumps due to change resistance in the ways processes are managed and decisions are made. present a significant barrier to adoption of the latest and greatest approaches.

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

CIO Business Intelligence

JULY 1, 2024

By documenting cases where automated systems misbehave, glitch or jeopardize users, we can better discern problematic patterns and mitigate risks. Real-time monitoring tools are essential, according to Luke Dash, CEO of risk management platform ISMS.online.

Reporting 129

Reporting Risk Management Experimentation Risk 129

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. Over time, using machine learning and AI, CBRE has managed to reduce manual lease processing times by 25% and cut positive false alarms in managed commercial facilities by 65%. And those experiments have paid off. And those experiments have paid off.

Experimentation 130

Experimentation Strategy Machine Learning Interactive 130

CIO Business Intelligence

APRIL 12, 2024

This anticipated move could completely transform how these companies hire new employees and how they manage and deliver the technology employees use. Right now most organizations tend to be in the experimental phases of using the technology to supplement employee tasks, but that is likely to change, and quickly, experts say.

Experimentation 143

Experimentation Reporting Testing Statistics 143

DataRobot Blog

JANUARY 10, 2023

Data science teams of all sizes need a productive, collaborative method for rapid AI experimentation. DataRobot Notebooks is a fully hosted and managed notebooks platform with auto-scaling compute capabilities so you can focus more on the data science and less on low-level infrastructure management. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52