Remove Experimentation Remove Metrics Remove Risk
article thumbnail

10 AI strategy questions every CIO must answer

CIO Business Intelligence

The time for experimentation and seeing what it can do was in 2023 and early 2024. Ethical, legal, and compliance preparedness helps companies anticipate potential legal issues and ethical dilemmas, safeguarding the company against risks and reputational damage, he says. What ROI will AI deliver?

Strategy 141
article thumbnail

Practical Skills for The AI Product Manager

O'Reilly on Data

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

From project to product: Architecting the future of enterprise technology

CIO Business Intelligence

Understanding and tracking the right software delivery metrics is essential to inform strategic decisions that drive continuous improvement. When tied directly to strategic objectives, software delivery metrics become business enablers, not just technical KPIs. This alignment sets the stage for how we execute our transformation.

article thumbnail

Escaping POC Purgatory: Evaluation-Driven Development for AI Systems

O'Reilly on Data

The coordination tax: LLM outputs are often evaluated by nontechnical stakeholders (legal, brand, support) not just for functionality, but for tone, appropriateness, and risk. ML apps needed to be developed through cycles of experimentation (as were no longer able to reason about how theyll behave based on software specs).

Testing 174
article thumbnail

What you need to know about product management for AI

O'Reilly on Data

Because it’s so different from traditional software development, where the risks are more or less well-known and predictable, AI rewards people and companies that are willing to take intelligent risks, and that have (or can develop) an experimental culture. And you, as the product manager, are caught between them.

article thumbnail

Where CIOs should place their 2025 AI bets

CIO Business Intelligence

CIOs feeling the pressure will likely seek more pragmatic AI applications, platform simplifications, and risk management practices that have short-term benefits while becoming force multipliers to longer-term financial returns. In 2024, departments and teams experimented with gen AI tools tied to their workflows and operating metrics.

article thumbnail

AI agents will transform business processes — and magnify risks

CIO Business Intelligence

Adding smarter AI also adds risk, of course. “At The big risk is you take the humans out of the loop when you let these into the wild.” Our goal is to analyze logs and metrics, connecting them with the source code to gain insights into code fixes, vulnerabilities, performance issues, and security concerns,” he says.

Risk 136