Remove Document Remove Experimentation Remove Risk
article thumbnail

Where CIOs should place their 2025 AI bets

CIO Business Intelligence

CIOs feeling the pressure will likely seek more pragmatic AI applications, platform simplifications, and risk management practices that have short-term benefits while becoming force multipliers to longer-term financial returns. CIOs should consider placing these five AI bets in 2025.

article thumbnail

Escaping POC Purgatory: Evaluation-Driven Development for AI Systems

O'Reilly on Data

The coordination tax: LLM outputs are often evaluated by nontechnical stakeholders (legal, brand, support) not just for functionality, but for tone, appropriateness, and risk. ML apps needed to be developed through cycles of experimentation (as were no longer able to reason about how theyll behave based on software specs).

Testing 174
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

CIOs to spend ambitiously on AI in 2025 — and beyond

CIO Business Intelligence

Nate Melby, CIO of Dairyland Power Cooperative, says the Midwestern utility has been churning out large language models (LLMs) that not only automate document summarization but also help manage power grids during storms, for example. The firm has also established an AI academy to train all its employees.

ROI 137
article thumbnail

Bringing an AI Product to Market

O'Reilly on Data

Without clarity in metrics, it’s impossible to do meaningful experimentation. AI PMs must ensure that experimentation occurs during three phases of the product lifecycle: Phase 1: Concept During the concept phase, it’s important to determine if it’s even possible for an AI product “ intervention ” to move an upstream business metric.

Marketing 364
article thumbnail

AI agents will transform business processes — and magnify risks

CIO Business Intelligence

Adding smarter AI also adds risk, of course. “At The big risk is you take the humans out of the loop when you let these into the wild.” When it comes to security, though, agentic AI is a double-edged sword with too many risks to count, he says. “We That means the projects are evaluated for the amount of risk they involve.

Risk 136
article thumbnail

Business Strategies for Deploying Disruptive Tech: Generative AI and ChatGPT

Rocket-Powered Data Science

3) How do we get started, when, who will be involved, and what are the targeted benefits, results, outcomes, and consequences (including risks)? encouraging and rewarding) a culture of experimentation across the organization. Encourage and reward a Culture of Experimentation that learns from failure, “ Test, or get fired!

Strategy 290
article thumbnail

Going ‘AI native’ with in-house ChatGPT the MITRE way

CIO Business Intelligence

And it enables research teams to analyze legislation and policy documents in record time, delivering plans for proposed changes to these critical agencies in a day rather than weeks. By June 2024, MITREChatGPT offered document analysis and reasoning on thousands of documents, provided an enterprise prompt library, and made GPT 3.5