Remove Experimentation Remove Software Remove Uncertainty
article thumbnail

Escaping POC Purgatory: Evaluation-Driven Development for AI Systems

O'Reilly on Data

Most teams approach this like traditional software development but quickly discover it’s a fundamentally different beast. Check out the graph belowsee how excitement for traditional software builds steadily while GenAI starts with a flashy demo and then hits a wall of challenges? Whats worse: Inputs are rarely exactly the same.

Testing 168
article thumbnail

What you need to know about product management for AI

O'Reilly on Data

If you’re already a software product manager (PM), you have a head start on becoming a PM for artificial intelligence (AI) or machine learning (ML). Why AI software development is different. Machine learning adds uncertainty. This shift requires a fundamental change in your software engineering practice.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Lessons from the field: How Generative AI is shaping software development in 2023

CIO Business Intelligence

Specifically, organizations are contemplating Generative AI’s impact on software development. While the potential of Generative AI in software development is exciting, there are still risks and guardrails that need to be considered. Generative AI has forced organizations to rethink how they work and what can and should be adjusted.

Software 119
article thumbnail

AI Product Management After Deployment

O'Reilly on Data

In traditional software engineering, precedent has been established for the transition of responsibility from development teams to maintenance, user operations, and site reliability teams. This distinction assumes a slightly different definition of debugging than is often used in software development. Monitoring.

article thumbnail

Machine Learning Product Management: Lessons Learned

Domino Data Lab

Unfortunately, a common challenge that many industry people face includes battling “ the model myth ,” or the perception that because their work includes code and data, their work “should” be treated like software engineering. These steps also reflect the experimental nature of ML product management.

article thumbnail

Towards optimal experimentation in online systems

The Unofficial Google Data Science Blog

Crucially, it takes into account the uncertainty inherent in our experiments. To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages. It is a big picture approach, worthy of your consideration.

article thumbnail

Why CIOs should invest in digital through economic headwinds

CIO Business Intelligence

Experiment with the “highly visible and highly hyped”: Gartner repeatedly pointed out that organisations that innovate during tough economic times “stay ahead of the pack”, with Mesaglio in particular calling for such experimentation to be public and visible.