O'Reilly on Data

MARCH 31, 2020

If you’re already a software product manager (PM), you have a head start on becoming a PM for artificial intelligence (AI) or machine learning (ML). Why AI software development is different. This shift requires a fundamental change in your software engineering practice.

Management 363

Management Machine Learning Experimentation Metrics 363

O'Reilly on Data

JUNE 11, 2024

It is important to be careful when deploying an AI application, but it’s also important to realize that all AI is experimental. It would have been very difficult to develop the expertise to build and train a model, and much more effective to work with a company that already has that expertise.

Modeling 340

Modeling Experimentation Interactive Data-driven 340

O'Reilly on Data

JANUARY 11, 2022

This trend started with the gigantic language model GPT-3. This may encourage the creation of more large-scale models; it might also drive a wedge between academic and industrial researchers. What does “reproducibility” mean if the model is so large that it’s impossible to reproduce experimental results?

Experimentation 361

Experimentation Modeling Strategy Machine Learning 361

CIO Business Intelligence

OCTOBER 24, 2024

in 2025, but software spending — four times larger than the data center segment — will grow by 14% next year, to $1.24 The software spending increases will be driven by several factors, including price increases, expanding license bases, and some AI investments , says John Lovelock, distinguished vice president analyst at Gartner.

IT 133

IT Data-driven Recreation/Entertainment Experimentation 133

O'Reilly on Data

SEPTEMBER 8, 2020

That cyclic process, which is about collaboration between software developers and customers, may be exactly what we need to get beyond the “AI as Oracle” interaction. Most AI systems we’ve seen envision AI as an oracle: you give it the input, it pops out the answer.

Experimentation 363

Experimentation Interactive Modeling Software 363

CIO Business Intelligence

OCTOBER 24, 2024

As they look to operationalize lessons learned through experimentation, they will deliver short-term wins and successfully play the gen AI — and other emerging tech — long game,” Leaver said. AI-driven software development hits snags Gen AI is becoming a pervasive force in all phases of software delivery.

ROI 127

ROI Data-driven Enterprise Experimentation 127

CIO Business Intelligence

JANUARY 14, 2025

Understanding and tracking the right software delivery metrics is essential to inform strategic decisions that drive continuous improvement. Wikipedia defines a software architect as a software expert who makes high-level design choices and dictates technical standards, including software coding standards, tools, and platforms.

Enterprise 119

Enterprise Technology Metrics Measurement 119

Occam's Razor

AUGUST 12, 2013

than multi-channel attribution modeling. We have fought valiant battles, paid expensive consultants, purchased a crazy amount of software, and achieved an implementation high that is quickly, followed by a " gosh darn it where is my return on investment from all this? Multi-Channel Attribution Models. Grab a Red Bull.

Modeling 162

Modeling Optimization Marketing Interactive 162

CIO Business Intelligence

MARCH 12, 2024

Generative AI is already having an impact on multiple areas of IT, most notably in software development. Still, gen AI for software development is in the nascent stages, so technology leaders and software teams can expect to encounter bumps in the road.

Software 131

Software Experimentation Testing Cost-Benefit 131

O'Reilly on Data

OCTOBER 13, 2020

In traditional software engineering, precedent has been established for the transition of responsibility from development teams to maintenance, user operations, and site reliability teams. This distinction assumes a slightly different definition of debugging than is often used in software development.

Management 314

Management Machine Learning Metrics Modeling 314

DataKitchen

APRIL 13, 2021

DataOps needs a directed graph-based workflow that contains all the data access, integration, model and visualization steps in the data analytic production process. ICEDQ — Software used to automate the testing of ETL/Data Warehouse and Data Migration. Liquibase — Database release automation for software development teams.

Testing 300

Testing Machine Learning Consulting Data Science 300

O'Reilly on Data

MARCH 18, 2020

Whether it’s controlling for common risk factors—bias in model development, missing or poorly conditioned data, the tendency of models to degrade in production—or instantiating formal processes to promote data governance, adopters will have their work cut out for them as they work to establish reliable AI production lines.

Enterprise 349

Enterprise Deep Learning Data Governance Risk 349

Cloudera

NOVEMBER 19, 2021

We recognise that experimentation is an important component of any enterprise machine learning practice. But, we also know that experimentation alone doesn’t yield business value. Organizations need to usher their ML models out of the lab (i.e., Organizations must think about an ML model in terms of its entire life cycle.

Machine Learning 122

Machine Learning Modeling Experimentation IT 122

O'Reilly on Data

MARCH 23, 2023

Maybe it’s surprising that ChatGPT can write software, maybe it isn’t; we’ve had over a year to get used to GitHub Copilot, which was based on an earlier version of GPT. What Software Are We Talking About? It’s important to understand that ChatGPT is not actually a language model. and 4 Large language models developed by OpenAI.

IT 348

IT Modeling Testing Risk 348

CIO Business Intelligence

SEPTEMBER 6, 2023

Since ChatGPT’s release in November of 2022, there have been countless conversations on the impact of similar large language models. Specifically, organizations are contemplating Generative AI’s impact on software development. Generative AI has forced organizations to rethink how they work and what can and should be adjusted.

Software 119

Software Risk Experimentation Data Lake 119

CIO Business Intelligence

MARCH 5, 2025

Two years of experimentation may have given rise to several valuable use cases for gen AI , but during the same period, IT leaders have also learned that the new, fast-evolving technology isnt something to jump into blindly. All the major software vendors are putting it into their products, he says. And if it does work, its all upside.

Testing 99

Testing Measurement Technology Experimentation 99

CIO Business Intelligence

MAY 23, 2022

Many companies whose AI model training infrastructure is not proximal to their data lake incur steeper costs as the data sets grow larger and AI models become more complex. The cloud is great for experimentation when data sets are smaller and model complexity is light. Potential headaches of DIY on-prem infrastructure.

Experimentation 137

Experimentation Cost-Benefit Data Lake Data Science 137

DataKitchen

JANUARY 20, 2021

A centralized team can publish a set of software services that support the rollout of Agile/DataOps. The center of excellence (COE) model leverages the DataOps team to solve real-world challenges. For example, some teams may recognize services revenue in the quarter booked, and others may amortize the revenue over the contract period.

Metrics 243

Metrics Experimentation Measurement Testing 243

AWS Big Data

DECEMBER 12, 2024

In the context of comprehensive data governance, Amazon DataZone offers organization-wide data lineage visualization using Amazon Web Services (AWS) services, while dbt provides project-level lineage through model analysis and supports cross-project integration between data lakes and warehouses.

Snapshot 79

Snapshot Recreation/Entertainment Experimentation Data Lake 79

CIO Business Intelligence

JULY 11, 2023

In recent years, we have witnessed a tidal wave of progress and excitement around large language models (LLMs) such as ChatGPT and GPT-4. On the contrary, the software can still be deployed with one click on any public or private cloud, managed, and scaled accordingly.

Modeling 95

Modeling Testing Experimentation Risk Management 95

Domino Data Lab

MAY 15, 2019

Unfortunately, a common challenge that many industry people face includes battling “ the model myth ,” or the perception that because their work includes code and data, their work “should” be treated like software engineering. These steps also reflect the experimental nature of ML product management.

Machine Learning 111

Machine Learning Management Experimentation Data Science 111

CIO Business Intelligence

AUGUST 20, 2024

Our mental models of what constitutes a high-performance team have evolved considerably over the past five years. Post-pandemic, high-performance teams excelled at remote and hybrid working models, were more empathetic to individual needs, and leveraged automation to reduce manual work.

IT 141

IT Digital Transformation Experimentation Risk 141

CIO Business Intelligence

SEPTEMBER 18, 2024

Even as it designs 3D generative AI models for future customer deployment, CAD/CAM design giant Autodesk is “leaning” into generative AI for its customer service operations, deploying Salesforce’s Einstein for Service with plans to use Agentforce in the future, CIO Prakash Kota says.

Digital Transformation 119

Digital Transformation Experimentation Sales Software 119

AWS Big Data

NOVEMBER 19, 2024

Customers maintain multiple MWAA environments to separate development stages, optimize resources, manage versions, enhance security, ensure redundancy, customize settings, improve scalability, and facilitate experimentation. This approach offers greater flexibility and control over workflow management. The introduction of mw1.micro

Metadata 106

Metadata Cost-Benefit Metrics Optimization 106

The Unofficial Google Data Science Blog

APRIL 23, 2024

Experiments, Parameters and Models At Youtube, the relationships between system parameters and metrics often seem simple — straight-line models sometimes fit our data well. That is true generally, not just in these experiments — spreading measurements out is generally better, if the straight-line model is a priori correct.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

DataRobot Blog

MARCH 8, 2023

Today, SAP and DataRobot announced a joint partnership to enable customers connect core SAP software, containing mission-critical business data, with the advanced Machine Learning capabilities of DataRobot to make more intelligent business predictions with advanced analytics.

Enterprise 105

Enterprise Experimentation Machine Learning Data Science 105

CIO Business Intelligence

JUNE 13, 2024

Proof that even the most rigid of organizations are willing to explore generative AI arrived this week when the US Department of the Air Force (DAF) launched an experimental initiative aimed at Guardians, Airmen, civilian employees, and contractors. It is not training the model, nor are responses refined based on any user inputs.

Testing 119

Testing Experimentation Data Processing Modeling 119

CIO Business Intelligence

MAY 21, 2024

With the generative AI gold rush in full swing, some IT leaders are finding generative AI’s first-wave darlings — large language models (LLMs) — may not be up to snuff for their more promising use cases. With this model, patients get results almost 80% faster than before. It’s fabulous.”

IT 131

IT Cost-Benefit Experimentation Forecasting 131

CIO Business Intelligence

DECEMBER 19, 2024

In especially high demand are IT pros with software development, data science and machine learning skills. In the EV and battery space, software engineers and product managers are driving the build-out of connected charging networks and improving battery life.

IT 52

IT IoT Machine Learning Optimization 52

Domino Data Lab

MARCH 20, 2019

Our mission at Domino is to enable organizations to put models at the heart of their business. Today we’re announcing two major new capabilities in Domino that make model development easier and faster for data scientists. This pain point is magnified in organizations with teams of data scientists working on numerous experiments.

Management 97

Management Experimentation Data Science Modeling 97

CIO Business Intelligence

MARCH 19, 2025

I first described the overall AI landscape and made sure they realized weve been doing AI for quite a while in the form of machine learning and other deterministic models. This enforces the need for good data governance, as AI models will surface incorrect data more frequently, and most likely at a greater cost to the business.

Metrics 64

Metrics Enterprise Cost-Benefit Experimentation 64

CIO Business Intelligence

AUGUST 14, 2024

It’s embedded in the applications we use every day and the security model overall is pretty airtight. Microsoft has also made investments beyond OpenAI, for example in Mistral and Meta’s LLAMA models, in its own small language models like Phi, and by partnering with providers like Cohere, Hugging Face, and Nvidia. That’s risky.”

Enterprise 136

Enterprise Cost-Benefit Experimentation Modeling 136

CIO Business Intelligence

OCTOBER 15, 2024

One example is how DevOps teams use feature flags, which can drive agile experimentation by enabling product managers to test features and user experience variants. Shifting operations earlier in the software development lifecycle increases cognitive load and decreases developer productivity.”

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

MAY 12, 2022

But there are deeper challenges because predictive analytics software can’t magically anticipate moments when the world shifts gears and the future bears little relationship to the past. Drag-and-drop Modeler for creating pipelines, IBM integrations. Driverless AI offers automated pipeline; AI adapts to incoming data. Open source core.

Predictive Analytics 137

Predictive Analytics Analytics Statistics Machine Learning 137

CIO Business Intelligence

APRIL 25, 2024

During the summer of 2023, at the height of the first wave of interest in generative AI, LinkedIn began to wonder whether matching candidates with employers and making feeds more useful would be better served with the help of large language models (LLMs). Generative AI, Software Development, Technology Industry

IT 139

IT Cost-Benefit Experimentation Metrics 139

CIO Business Intelligence

MARCH 7, 2023

While many organizations are successful with agile and Scrum, and I believe agile experimentation is the cornerstone of driving digital transformation, there isn’t a one-size-fits-all approach. CIOs should consider technologies that promote their hybrid working models to replace in-person meetings.

Digital Transformation 145

Digital Transformation Data Governance Machine Learning Experimentation 145

CIO Business Intelligence

JULY 1, 2024

Notable examples of AI safety incidents include: Trading algorithms causing market “flash crashes” ; Facial recognition systems leading to wrongful arrests ; Autonomous vehicle accidents ; AI models providing harmful or misleading information through social media channels.

Reporting 129

Reporting Risk Management Experimentation Risk 129