O'Reilly on Data

JULY 28, 2020

Without clarity in metrics, it’s impossible to do meaningful experimentation. AI PMs must ensure that experimentation occurs during three phases of the product lifecycle: Phase 1: Concept During the concept phase, it’s important to determine if it’s even possible for an AI product “ intervention ” to move an upstream business metric.

Marketing 363

Marketing Experimentation Metrics Testing 363

CIO Business Intelligence

JULY 11, 2023

In recent years, we have witnessed a tidal wave of progress and excitement around large language models (LLMs) such as ChatGPT and GPT-4. By deploying the LLM within their own VPC, the company can benefit from the AI’s insights without risking the exposure of their valuable data.

Modeling 84

Modeling Testing Experimentation Risk Management 84

CIO Business Intelligence

NOVEMBER 13, 2024

The new version, ARIS 10 SR27, available now, includes AI Companion, which a release stated contains capabilities such as the ability for users to query information stored in models within the ARIS repository without the need for an exact match on keywords, and can translate text-based descriptions into structured BPM models.

Experimentation 108

Experimentation Testing Software Risk 108

CIO Business Intelligence

JULY 5, 2024

So, to maximize the ROI of gen AI efforts and investments, it’s important to move from ad-hoc experimentation to a more purposeful strategy and systematic approach to implementation. Define which strategic themes relate to your business model, processes, products, and services. This may impact some of your vendor selections as well.

ROI 123

ROI Experimentation Risk Cost-Benefit 123

O'Reilly on Data

MAY 14, 2020

AI PMs should enter feature development and experimentation phases only after deciding what problem they want to solve as precisely as possible, and placing the problem into one of these categories. Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model.

Management 363

Management Experimentation B2B Machine Learning 363

CIO Business Intelligence

SEPTEMBER 12, 2024

The familiar narrative illustrates the double-edged sword of “shadow AI”—technologies used to accomplish AI-powered tasks without corporate approval or oversight, bringing quick wins but potentially exposing organizations to significant risks. Generative AI models can perpetuate and amplify biases in training data when constructing output.

Experimentation 111

Experimentation Risk Cost-Benefit Strategy 111

CIO Business Intelligence

SEPTEMBER 23, 2024

From the rise of value-based payment models to the upheaval caused by the pandemic to the transformation of technology used in everything from risk stratification to payment integrity, radical change has been the only constant for health plans. The last decade has seen its fair share of volatility in the healthcare industry.

Management 114

Management Experimentation Strategy Data-driven 114

CIO Business Intelligence

AUGUST 20, 2024

Our mental models of what constitutes a high-performance team have evolved considerably over the past five years. Post-pandemic, high-performance teams excelled at remote and hybrid working models, were more empathetic to individual needs, and leveraged automation to reduce manual work. What is a high-performance team today?

IT 140

IT Digital Transformation Experimentation Risk 140

CIO Business Intelligence

OCTOBER 3, 2024

From budget allocations to model preferences and testing methodologies, the survey unearths the areas that matter most to large, medium, and small companies, respectively. The complexity and scale of operations in large organizations necessitate robust testing frameworks to mitigate these risks and remain compliant with industry regulations.

Testing 126

Testing Cost-Benefit Experimentation Risk 126

CIO Business Intelligence

NOVEMBER 4, 2024

IDC, for instance, recommends the NIST AI Risk Management Framework as a suitable standard to help CIOs develop AI governance in house, as well as EU AI ACT provisions, says Trinidad, who cites best practices for some aspects of AI governance in “ IDC PeerScape: Practices for Securing AI Models and Applications.” The challenges? “AI

Uncertainty 133

Uncertainty Cost-Benefit Risk Experimentation 133

DataRobot Blog

DECEMBER 6, 2022

Other organizations are just discovering how to apply AI to accelerate experimentation time frames and find the best models to produce results. Taking a Multi-Tiered Approach to Model Risk Management. How Model Observability Provides a 360° View of Models in Production. Read the blog. Read the blog.

Experimentation 145

Experimentation Forecasting Data-driven Machine Learning 145

Rocket-Powered Data Science

FEBRUARY 15, 2023

While generative AI has been around for several years , the arrival of ChatGPT (a conversational AI tool for all business occasions, built and trained from large language models) has been like a brilliant torch brought into a dark room, illuminating many previously unseen opportunities.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

CIO Business Intelligence

OCTOBER 15, 2024

Recommendation : CIOs should adopt a risk-informed approach, understanding business, customer, and employee impacts before setting application-specific continuous deployment strategies. Shortchanging end-user and developer experiences Many DevOps practices focus on automation, such as CI/CD and infrastructure as code.

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

OCTOBER 3, 2024

Rather than pull away from big iron in the AI era, Big Blue is leaning into it, with plans in 2025 to release its next-generation Z mainframe , with a Telum II processor and Spyre AI Accelerator Card, positioned to run large language models (LLMs) and machine learning models for fraud detection and other use cases.

Machine Learning 135

Machine Learning Experimentation Insurance Modeling 135

CIO Business Intelligence

AUGUST 5, 2024

Healthcare Domain Expertise: It cannot be said enough that anyone developing AI-driven models for healthcare needs to understand the unique use cases and stringent data security and privacy requirements – and the detailed nuances of how this information will be used – in the specific healthcare setting where the technology will be deployed.

Experimentation 120

Experimentation Risk Metadata Data-driven 120

IBM Big Data Hub

JUNE 10, 2024

As organizations strive to harness the power of AI while controlling costs, leveraging anything as a service (XaaS) models emerges as a strategic approach. Embracing the power of XaaS XaaS encompasses a broad spectrum of cloud-based and on-premises service models that offer scalable and cost-effective solutions to businesses.

Experimentation 62

Experimentation Optimization Risk Management Machine Learning 62

Decision Management Solutions

DECEMBER 17, 2021

In the recent McKinsey article discussing designing next-generation credit-decisioning models they outlined four best practices for automated credit-decisioning models for banks as they continue their digital transformations. Digital lending based on high-performance credit-decisioning models, says McKinsey, lead to: Increased revenue.

Modeling 52

Modeling Visualization Experimentation Risk 52

CIO Business Intelligence

OCTOBER 14, 2024

Taylor adds that functional CIOs tend to concentrate on business-as-usual facets of IT such as system and services reliability; cost reduction and improving efficiency; risk management/ensuring the security and reliability of IT systems; and ongoing support of existing technology and tracking daily metrics.

Digital Transformation 131

Digital Transformation Experimentation Cost-Benefit Risk 131

CIO Business Intelligence

JULY 1, 2024

Notable examples of AI safety incidents include: Trading algorithms causing market “flash crashes” ; Facial recognition systems leading to wrongful arrests ; Autonomous vehicle accidents ; AI models providing harmful or misleading information through social media channels.

Reporting 125

Reporting Risk Management Experimentation Risk 125

CIO Business Intelligence

AUGUST 15, 2024

Many of those gen AI projects will fail because of poor data quality, inadequate risk controls, unclear business value , or escalating costs , Gartner predicts. When we do planning sessions with our clients, two thirds of the solutions they need don’t necessarily fit the generative AI model.

ROI 142

ROI Cost-Benefit Experimentation Deep Learning 142

CIO Business Intelligence

JUNE 13, 2024

Proof that even the most rigid of organizations are willing to explore generative AI arrived this week when the US Department of the Air Force (DAF) launched an experimental initiative aimed at Guardians, Airmen, civilian employees, and contractors. It is not training the model, nor are responses refined based on any user inputs.

Testing 119

Testing Experimentation Data Processing Modeling 119

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. Davé and his team’s achievements in AI are due in large part to creating opportunities for experimentation — and ensuring those experiments align with CBRE’s business strategy. Let’s start with the models. And those experiments have paid off.

Experimentation 129

Experimentation Strategy Machine Learning Interactive 129

CIO Business Intelligence

JULY 12, 2024

“Not only does this particular low-code solution make rapid experimentation possible, it also offers orchestration capabilities so we can plug different services in and out very quickly,” says Pacynski. And if it doesn’t work, we have the flexibility to take out a component and put it in something different without any hassle.”

Experimentation 124

Experimentation Testing Enterprise Strategy 124

CIO Business Intelligence

AUGUST 23, 2024

Data scientists at Bayer have developed several proofs of concept of generative AI models on the new platform that remain in discovery and evaluation phase for “efficacy,” McQueen says, adding that the models won’t be in production until 2025. The R&D pipeline is pretty highly confidential at this point,” he says. It’s additive.”

Data Science 100

Data Science Experimentation Testing Modeling 100

Cloudera

OCTOBER 8, 2024

This service supports a range of optimized AI models, enabling seamless and scalable AI inference. By 2023, the focus shifted towards experimentation. Enterprise developers began exploring proof of concepts (POCs) for generative AI applications, leveraging API services and open models such as Llama 2 and Mistral.

Optimization 74

Optimization Experimentation Metrics Modeling 74

DataRobot Blog

SEPTEMBER 30, 2022

How do you track the integrity of a machine learning model in production? Model Observability can help. By tracking service, drift, prediction data, training data, and custom metrics, you can keep your models and predictions relevant in a fast-changing world. Model Observability Features.

Slice and Dice 52

Slice and Dice Modeling Metrics Experimentation 52

CIO Business Intelligence

NOVEMBER 21, 2023

Ask IT leaders about their challenges with shadow IT, and most will cite the kinds of security, operational, and integration risks that give shadow IT its bad rep. That’s not to downplay the inherent risks of shadow IT.

IT 137

IT Risk Cost-Benefit Risk Management 137

CIO Business Intelligence

JULY 24, 2024

Key strategies for exploration: Experimentation: Conduct small-scale experiments. Adobe’s transition from packaged software to the Creative Cloud service model is an example of a strategic move to exploit new market opportunities and scale for success with recurrent revenue and a broader user base.

Technology 97

Technology Experimentation Cost-Benefit ROI 97

O'Reilly on Data

MARCH 18, 2020

Whether it’s controlling for common risk factors—bias in model development, missing or poorly conditioned data, the tendency of models to degrade in production—or instantiating formal processes to promote data governance, adopters will have their work cut out for them as they work to establish reliable AI production lines.

Enterprise 314

Enterprise Deep Learning Data Governance Risk 314

CIO Business Intelligence

JUNE 21, 2024

The AI data center pod will also be used to power MITRE’s federal AI sandbox and testbed experimentation with AI-enabled applications and large language models (LLMs). We took a risk. API available to projects, Cenkl says.

Experimentation 131

Experimentation Data mining Reporting Data Processing 131

CIO Business Intelligence

APRIL 29, 2024

As organizations roll out AI applications and AI-enabled smartphones and devices, IT leaders may need to sell the benefits to employees or risk those investments falling short of business expectations. They need to have a culture of experimentation.” CIOs should be “change agents” who “embrace the art of the possible,” he says.

Forecasting 135

Forecasting Experimentation Risk Consulting 135

CIO Business Intelligence

AUGUST 2, 2023

The survey found tension between business leaders seeking competitive advantage, and IT leaders wanting to limit risks. Interestingly, non-IT leaders were more likely to report actively using generative AI (73%) than IT leaders (59%), suggesting there’s plenty of experimentation going on beyond the purview of the IT department.

Enterprise 98

Enterprise Experimentation Manufacturing Risk 98

O'Reilly on Data

APRIL 4, 2024

The race to the top is no longer driven by who has the best product or the best business model, but by who has the blessing of the venture capitalists with the deepest pockets—a blessing that will allow them to acquire the most customers the most quickly, often by providing services below cost. That is true product-market fit.

Marketing 156

Marketing Finance Modeling Experimentation 156

Smart Data Collective

MAY 22, 2022

The decisions are based on extensive experimentation and research to improve effectiveness without altering customer experience. With AI, the risk score for a device doesn’t depend on individual indicators. Predicting If a Device Is at Risk. Therefore, the risk score is always being adjusted accordingly.

Data-driven 135

Data-driven Risk Experimentation IT 135

CIO Business Intelligence

SEPTEMBER 6, 2023

Since ChatGPT’s release in November of 2022, there have been countless conversations on the impact of similar large language models. While the potential of Generative AI in software development is exciting, there are still risks and guardrails that need to be considered. To learn more, visit us here.

Software 119

Software Experimentation Risk Uncertainty 119

CIO Business Intelligence

DECEMBER 19, 2023

Many technology investments are merely transitionary, taking something done today and upgrading it to a better capability without necessarily transforming the business or operating model. If CIOs don’t improve conversions from pilot to production, they may find their investors losing patience in the process and culture of experimentation.

Digital Transformation 143

Digital Transformation Unstructured Data Machine Learning Risk Management 143

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data.

Management 338

Management Machine Learning Experimentation Metrics 338