O'Reilly on Data

MARCH 18, 2020

Whether it’s controlling for common risk factors—bias in model development, missing or poorly conditioned data, the tendency of models to degrade in production—or instantiating formal processes to promote data governance, adopters will have their work cut out for them as they work to establish reliable AI production lines.

Enterprise 349

Enterprise Deep Learning Data Governance Risk 349

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data.

Management 363

Management Machine Learning Experimentation Metrics 363

CIO Business Intelligence

JANUARY 21, 2025

CIOs feeling the pressure will likely seek more pragmatic AI applications, platform simplifications, and risk management practices that have short-term benefits while becoming force multipliers to longer-term financial returns. CIOs should consider placing these five AI bets in 2025.

Cost-Benefit 142

Cost-Benefit Data-driven Strategy Marketing 142

Rocket-Powered Data Science

FEBRUARY 15, 2023

While generative AI has been around for several years , the arrival of ChatGPT (a conversational AI tool for all business occasions, built and trained from large language models) has been like a brilliant torch brought into a dark room, illuminating many previously unseen opportunities.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

O'Reilly on Data

APRIL 19, 2021

Relatively few respondents are using version control for data and models. Tools for versioning data and models are still immature, but they’re critical for making AI results reproducible and reliable. The biggest skills gaps were ML modelers and data scientists (52%), understanding business use cases (49%), and data engineering (42%).

Enterprise 364

Enterprise Risk Modeling Manufacturing 364

CIO Business Intelligence

OCTOBER 24, 2024

As they look to operationalize lessons learned through experimentation, they will deliver short-term wins and successfully play the gen AI — and other emerging tech — long game,” Leaver said. Determining the optimal level of autonomy to balance risk and efficiency will challenge business leaders,” Le Clair said.

ROI 127

ROI Data-driven Enterprise Experimentation 127

CIO Business Intelligence

AUGUST 21, 2024

According to Gartner, an agent doesn’t have to be an AI model. Starting in 2018, the agency used agents, in the form of Raspberry PI computers running biologically-inspired neural networks and time series models, as the foundation of a cooperative network of sensors. “It Adding smarter AI also adds risk, of course. “At

Risk 136

Risk Insurance Cost-Benefit Software 136

O'Reilly on Data

SEPTEMBER 15, 2020

In my book, I introduce the Technical Maturity Model: I define technical maturity as a combination of three factors at a given point of time. Technical competence results in reduced risk and uncertainty. AI initiatives may also require significant considerations for governance, compliance, ethics, cost, and risk.

Advertising 357

Advertising Cost-Benefit ROI Machine Learning 357

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, The GPT-series LLMs are also called “foundation models.” GPT-2, 3, 3.5,

IT 344

IT Modeling Testing Risk 344

DataRobot Blog

DECEMBER 6, 2022

Other organizations are just discovering how to apply AI to accelerate experimentation time frames and find the best models to produce results. Taking a Multi-Tiered Approach to Model Risk Management. How Model Observability Provides a 360° View of Models in Production. Read the blog. Read the blog.

Experimentation 145

Experimentation Forecasting Data-driven Machine Learning 145

CIO Business Intelligence

NOVEMBER 7, 2024

It covers essential topics like artificial intelligence, our use of data models, our approach to technical debt, and the modernization of legacy systems. This team addresses potential risks, manages AI across the company, provides guidance, implements necessary training, and keeps abreast of emerging regulatory changes.

Insurance 120

Insurance Experimentation Testing Technology 120

CIO Business Intelligence

MARCH 5, 2025

Two years of experimentation may have given rise to several valuable use cases for gen AI , but during the same period, IT leaders have also learned that the new, fast-evolving technology isnt something to jump into blindly. Use a mix of established and promising small players To mitigate risk, Gupta rarely uses small vendors on big projects.

Testing 82

Testing Measurement Technology Experimentation 82

O'Reilly on Data

APRIL 4, 2024

The race to the top is no longer driven by who has the best product or the best business model, but by who has the blessing of the venture capitalists with the deepest pockets—a blessing that will allow them to acquire the most customers the most quickly, often by providing services below cost. That is true product-market fit.

Marketing 226

Marketing Finance Modeling Experimentation 226

Domino Data Lab

AUGUST 6, 2021

Model Risk Management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

CIO Business Intelligence

AUGUST 20, 2024

Our mental models of what constitutes a high-performance team have evolved considerably over the past five years. Post-pandemic, high-performance teams excelled at remote and hybrid working models, were more empathetic to individual needs, and leveraged automation to reduce manual work. What is a high-performance team today?

IT 141

IT Digital Transformation Experimentation Risk 141

CIO Business Intelligence

JULY 5, 2024

So, to maximize the ROI of gen AI efforts and investments, it’s important to move from ad-hoc experimentation to a more purposeful strategy and systematic approach to implementation. Define which strategic themes relate to your business model, processes, products, and services. This may impact some of your vendor selections as well.

ROI 124

ROI Experimentation Risk Cost-Benefit 124

Smart Data Collective

MAY 22, 2022

The decisions are based on extensive experimentation and research to improve effectiveness without altering customer experience. With AI, the risk score for a device doesn’t depend on individual indicators. Predicting If a Device Is at Risk. Therefore, the risk score is always being adjusted accordingly.

Data-driven 145

Data-driven Risk Experimentation Technology 145

CIO Business Intelligence

JULY 11, 2023

In recent years, we have witnessed a tidal wave of progress and excitement around large language models (LLMs) such as ChatGPT and GPT-4. By deploying the LLM within their own VPC, the company can benefit from the AI’s insights without risking the exposure of their valuable data.

Modeling 95

Modeling Testing Experimentation Risk Management 95

CIO Business Intelligence

MARCH 19, 2025

I first described the overall AI landscape and made sure they realized weve been doing AI for quite a while in the form of machine learning and other deterministic models. This can cause risk without a clear business case. I then described what I think of as the three categories of generative AI. Thats gen AI driving revenue.

Metrics 64

Metrics Enterprise Cost-Benefit Experimentation 64

Occam's Razor

NOVEMBER 17, 2008

There is a tendency to think experimentation and testing is optional. 4 Big Bets, Low Risks, Happy Customers. You have just launched something risky, yet you have controlled the risk by reducing exposure of the risky idea. You can control the risk you want to take. # But remember you can control risk.

Testing 113

Testing Experimentation Risk Optimization 113

Domino Data Lab

MAY 15, 2019

Unfortunately, a common challenge that many industry people face includes battling “ the model myth ,” or the perception that because their work includes code and data, their work “should” be treated like software engineering. These steps also reflect the experimental nature of ML product management.

Machine Learning 111

Machine Learning Management Experimentation Uncertainty 111

CIO Business Intelligence

OCTOBER 3, 2024

From budget allocations to model preferences and testing methodologies, the survey unearths the areas that matter most to large, medium, and small companies, respectively. The complexity and scale of operations in large organizations necessitate robust testing frameworks to mitigate these risks and remain compliant with industry regulations.

Testing 129

Testing Cost-Benefit Experimentation Risk 129

The Unofficial Google Data Science Blog

APRIL 23, 2024

Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. That is true generally, not just in these experiments — spreading measurements out is generally better, if the straight-line model is a priori correct.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

CIO Business Intelligence

SEPTEMBER 12, 2024

The familiar narrative illustrates the double-edged sword of “shadow AI”—technologies used to accomplish AI-powered tasks without corporate approval or oversight, bringing quick wins but potentially exposing organizations to significant risks. Generative AI models can perpetuate and amplify biases in training data when constructing output.

Experimentation 111

Experimentation Risk Cost-Benefit Strategy 111

CIO Business Intelligence

SEPTEMBER 23, 2024

From the rise of value-based payment models to the upheaval caused by the pandemic to the transformation of technology used in everything from risk stratification to payment integrity, radical change has been the only constant for health plans. The last decade has seen its fair share of volatility in the healthcare industry.

Management 115

Management Experimentation Data-driven Strategy 115

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. Davé and his team’s achievements in AI are due in large part to creating opportunities for experimentation — and ensuring those experiments align with CBRE’s business strategy. Let’s start with the models. And those experiments have paid off.

Experimentation 130

Experimentation Strategy Machine Learning Interactive 130

CIO Business Intelligence

AUGUST 5, 2024

Healthcare Domain Expertise: It cannot be said enough that anyone developing AI-driven models for healthcare needs to understand the unique use cases and stringent data security and privacy requirements – and the detailed nuances of how this information will be used – in the specific healthcare setting where the technology will be deployed.

Experimentation 124

Experimentation Risk Metadata Data-driven 124

DataRobot Blog

JANUARY 10, 2023

Most, if not all, machine learning (ML) models in production today were born in notebooks before they were put into production. Data science teams of all sizes need a productive, collaborative method for rapid AI experimentation. Capabilities Beyond Classic Jupyter for End-to-end Experimentation. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

CIO Business Intelligence

JULY 1, 2024

Notable examples of AI safety incidents include: Trading algorithms causing market “flash crashes” ; Facial recognition systems leading to wrongful arrests ; Autonomous vehicle accidents ; AI models providing harmful or misleading information through social media channels.

Reporting 129

Reporting Risk Management Experimentation Risk 129

CIO Business Intelligence

AUGUST 15, 2024

Many of those gen AI projects will fail because of poor data quality, inadequate risk controls, unclear business value , or escalating costs , Gartner predicts. When we do planning sessions with our clients, two thirds of the solutions they need don’t necessarily fit the generative AI model.

ROI 143

ROI Cost-Benefit Experimentation Deep Learning 143

CIO Business Intelligence

OCTOBER 3, 2024

Rather than pull away from big iron in the AI era, Big Blue is leaning into it, with plans in 2025 to release its next-generation Z mainframe , with a Telum II processor and Spyre AI Accelerator Card, positioned to run large language models (LLMs) and machine learning models for fraud detection and other use cases.

Machine Learning 140

Machine Learning Experimentation Insurance Modeling 140

CIO Business Intelligence

SEPTEMBER 10, 2024

We envisioned harnessing this data through predictive models to gain valuable insights into various aspects of the industry. Additionally, we explored how predictive models could be used to identify the ideal profile for haul truck drivers, with the goal of reducing accidents and fatalities. Failing is managing risk.

IT 122

IT Risk Risk Management Predictive Modeling 122

Corinium

JUNE 6, 2019

Regulations and compliance requirements, especially around pricing, risk selection, etc., Beyond that, we recommend setting up the appropriate data management and engineering framework including infrastructure, harmonization, governance, toolset strategy, automation, and operating model. In addition, the traditional challenges remain.

Analytics 250

Analytics Insurance Forecasting Deep Learning 250

CIO Business Intelligence

OCTOBER 15, 2024

Recommendation : CIOs should adopt a risk-informed approach, understanding business, customer, and employee impacts before setting application-specific continuous deployment strategies. Shortchanging end-user and developer experiences Many DevOps practices focus on automation, such as CI/CD and infrastructure as code.

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

MARCH 7, 2023

But the faster transition often caused underperforming apps, greater security risks, higher costs, and fewer business outcomes, forcing IT to address these issues before starting app modernizations. CIOs should consider technologies that promote their hybrid working models to replace in-person meetings.

Digital Transformation 145

Digital Transformation Data Governance Machine Learning Experimentation 145

CIO Business Intelligence

DECEMBER 11, 2024

Research from IDC predicts that we will move from the experimentation phase, the GenAI scramble that we saw in 2023 and 2024, and mature into the adoption phase in 2025/26 before moving into AI-fuelled businesses in 2027 and beyond. So what are the leaders doing differently?

ROI 59

ROI Experimentation Insurance Strategy 59

CIO Business Intelligence

NOVEMBER 21, 2023

Ask IT leaders about their challenges with shadow IT, and most will cite the kinds of security, operational, and integration risks that give shadow IT its bad rep. That’s not to downplay the inherent risks of shadow IT.

IT 137

IT Risk Cost-Benefit Data Science 137