DataRobot Blog

MAY 26, 2022

Last time , we discussed the steps that a modeler must pay attention to when building out ML models to be utilized within the financial institution. In summary, to ensure that they have built a robust model, modelers must make certain that they have designed the model in a way that is backed by research and industry-adopted practices.

Risk 52

Risk Modeling Metrics Business Objectives 52

Cloudera

OCTOBER 20, 2021

As part of these efforts, disclosure requirements will mandate that firms provide “the impact of a company’s activities on the environment and society, as well as the business and financial risks faced by a company due to its sustainability exposures.” What are the key climate risk measurements and impacts? They need to understand;

Risk 79

Risk Modeling Risk Management Measurement 79

Cloudera

JUNE 29, 2023

To ensure the stability of the US financial system, the implementation of advanced liquidity risk models and stress testing using (MI/AI) could potentially serve as a protective measure. To improve the way they model and manage risk, institutions must modernize their data management and data governance practices.

Data Architecture 85

Data Architecture Risk Management Risk Management 85

DataKitchen

JUNE 1, 2022

Quality/Tests/Trust. How many tests do I have in production? T ools/Models/Dashboards. Is my model still accurate? What is the average number of tests per pipeline? How many models and dashboards were deployed? Testing/Impact/Regressions. How many tests ran in the QA environment? Root Cause.

Risk 130

Risk Management Testing Dashboards 130

O'Reilly on Data

JUNE 19, 2019

A look at the landscape of tools for building and deploying robust, production-ready machine learning models. We are also beginning to see researchers share sample code written in popular open source libraries, and some even share pre-trained models. Model development. Model governance. Source: Ben Lorica.

Modeling 195

Modeling Machine Learning Testing Metrics 195

DataRobot Blog

JULY 22, 2022

Using AI-based models increases your organization’s revenue, improves operational efficiency, and enhances client relationships. You need to know where your deployed models are, what they do, the data they use, the results they produce, and who relies upon their results. That requires a good model governance framework.

Modeling 105

Modeling Risk Management Testing Risk 105

IBM Big Data Hub

MAY 13, 2024

In the previous blog , we discussed the differentiated approach by IBM to delivering enterprise-grade models. In this blog, we delve into why foundation model choices matter and how they empower businesses to scale gen AI with confidence. Why are model choices important?

Modeling 70

Modeling Cost-Benefit Risk Testing 70

David Menninger's Analyst Perspectives

SEPTEMBER 24, 2024

erroneous results), and an equal amount (32%) mentioned legal risk. AI governance should address a number of issues, including data privacy, bias in data and models, drift in model accuracy, hallucinations and toxicity. Red-teaming is a term used to describe human testing of models for vulnerabilities.

Testing 130

Testing Modeling Enterprise Risk 130

CIO Business Intelligence

AUGUST 20, 2024

million to fine-tune gen AI models, and $20 million to build custom models from scratch, according to recent estimates from Gartner. Most SMBs don’t have the resources to create and maintain their own AI models, and they will need to work with partners to run AI models, he adds.

Risk 131

Risk Cost-Benefit Insurance ROI 131

Cloudera

MARCH 2, 2022

Risk management is a highly dynamic discipline these days. Stress testing is a particular area that has become even more important throughout the pandemic. Similarly, the European Central Bank is issuing stress testing requirements related to climate risk given the potential economic shifts related to addressing climate change.

Testing 95

Testing Management Risk Management Forecasting 95

O'Reilly on Data

OCTOBER 30, 2023

While I am heartened to hear that the Executive Order on AI uses the Defense Production Act to compel disclosure of various data from the development of large AI models, these disclosures do not go far enough. These include: What data sources the model is trained on. Operational Metrics.

Measurement 304

Measurement Risk Metrics Modeling 304

Domino Data Lab

AUGUST 6, 2021

All models require testing and auditing throughout their deployment and, because models are continually learning, there is always an element of risk that they will drift from their original standards. As such, model governance needs to be applied to each model for as long as it’s being used.

Machine Learning 145

Machine Learning Modeling Testing Data Science 145

IBM Big Data Hub

MAY 8, 2023

This blog continues the discussion, now investigating the risks of adopting AI and proposes measures for a safe and judicious response to adopting AI. Risk and limitations of AI The risk associated with the adoption of AI in insurance can be separated broadly into two categories—technological and usage.

Insurance 68

Insurance Risk Testing Data Quality 68

O'Reilly on Data

DECEMBER 12, 2019

Not least is the broadening realization that ML models can fail. And that’s why model debugging, the art and science of understanding and fixing problems in ML models, is so critical to the future of ML. Because all ML models make mistakes, everyone who cares about ML should also care about model debugging. [1]

Machine Learning 197

Machine Learning Modeling Testing Risk Management 197

Domino Data Lab

AUGUST 6, 2021

Model Risk Management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

Cloudera

JULY 15, 2021

We recently hosted a roundtable focused on o ptimizing risk and exposure management with data insights. For financial institutions and insurers, risk and exposure management has always been a fundamental tenet of the business. Now, risk management has become exponentially complicated in multiple dimensions. . Area such as: .

Risk 99

Risk Optimization Management Insurance 99

CIO Business Intelligence

JULY 11, 2023

In recent years, we have witnessed a tidal wave of progress and excitement around large language models (LLMs) such as ChatGPT and GPT-4. By deploying the LLM within their own VPC, the company can benefit from the AI’s insights without risking the exposure of their valuable data.

Modeling 94

Modeling Testing Experimentation Risk Management 94

CIO Business Intelligence

JANUARY 19, 2023

Giving up control: rewards outweigh the risks The benefits of greater delegation include an increase in the velocity of decision making and an increased sense of ownership and accountability, says Cisco Sanchez, SVP and CIO at Qualcomm, which has been increasing the pace of delegation. They just need visibility.

Testing 134

Testing Digital Transformation Risk Management 134

CIO Business Intelligence

JULY 19, 2022

Phishing/pharming, card testing, identity theft, and first-party misuse remain the most prevalent fraud attacks, each affecting more than three in 10 merchants globally, according to 2022 Global Fraud Report , by the Merchants Risk Council, Cybersource, and Verifi. For the second consecutive year, fraud costs rose around the globe. [2].

Risk 98

Risk Machine Learning Reporting Sales 98

IBM Big Data Hub

SEPTEMBER 7, 2023

Today we are announcing our latest addition: a new family of IBM-built foundation models which will be available in watsonx.ai , our studio for generative AI, foundation models and machine learning. Collectively named “Granite,” these multi-size foundation models apply generative AI to both language and code.

Modeling 101

Modeling Risk Unstructured Data Enterprise 101

O'Reilly on Data

NOVEMBER 28, 2023

And everyone has opinions about how these language models and art generation programs are going to change the nature of work, usher in the singularity, or perhaps even doom the human race. 16% of respondents working with AI are using open source models. A few have even tried out Bard or Claude, or run LLaMA 1 on their laptop.

Enterprise 277

Enterprise Modeling Testing Reporting 277

CIO Business Intelligence

OCTOBER 3, 2024

From budget allocations to model preferences and testing methodologies, the survey unearths the areas that matter most to large, medium, and small companies, respectively. Healthcare-specific task-oriented models were also highly favored, with more than half (57%) of respondents using these models.

Testing 129

Testing Cost-Benefit Experimentation Risk 129

CIO Business Intelligence

MAY 17, 2022

Even if the AI apocalypse doesn’t come to pass, shortchanging AI ethics poses big risks to society — and to the enterprises that deploy those AI systems. The following real-world implementation issues highlight prominent risks every IT leader must account for in putting together their company’s AI deployment strategy.

Risk 145

Risk Data Science Consulting Machine Learning 145

CIO Business Intelligence

SEPTEMBER 10, 2024

The US Department of Commerce’s Bureau of Industry and Security (BIS) plans to introduce mandatory reporting requirements for developers of advanced AI models and cloud computing providers.

Reporting 131

Reporting Measurement Testing Risk 131

CIO Business Intelligence

NOVEMBER 19, 2024

Data privacy and compliance issues Failing: Mismanagement of internal data with external models can lead to privacy breaches and non-compliance with regulations. Bias and hallucinations Failing: GenAI models often produce biased or inaccurate outputs, leading to misinformation and potential legal issues.

Consulting 123

Consulting Cost-Benefit Experimentation Optimization 123

CIO Business Intelligence

MARCH 25, 2024

Modern digital organisations tend to use an agile approach to delivery, with cross-functional teams, product-based operating models , and persistent funding. But to deliver transformative initiatives, CIOs need to embrace the agile, product-based approach, and that means convincing the CFO to switch to a persistent funding model.

Modeling 124

Modeling Cost-Benefit IT ROI 124

CIO Business Intelligence

NOVEMBER 13, 2024

The new version, ARIS 10 SR27, available now, includes AI Companion, which a release stated contains capabilities such as the ability for users to query information stored in models within the ARIS repository without the need for an exact match on keywords, and can translate text-based descriptions into structured BPM models.

Experimentation 113

Experimentation Testing Software Risk 113

DataKitchen

MAY 10, 2024

It highlights how DataKitchen’s Data Observation solutions equip organizations to enhance their development practices, reduce deployment risks, and increase overall productivity. Each addition or modification poses potential risks that could propagate errors into production environments. How Many Tests Ran In The Qa Environment?

Testing 126

Testing Cost-Benefit Data Quality Dashboards 126

Cloudera

FEBRUARY 4, 2021

Financial services institutions need the ability to analyze and act on massive volumes of data from diverse sources in order to monitor, model, and manage risk across the enterprise. They need a comprehensive data and analytics platform to model risk exposures on-demand. Cloudera is that platform.

Risk 87

Risk Marketing Technology Management 87

Sisense

SEPTEMBER 23, 2019

If it breaks, it puts at risk, not just a single application, but an entire chain of business processes built around it. Once we have a solid foundation of unit tests which cover individual functions, API tests provide higher reliability covering an interface closer to the user, yet without the brittleness of UI tests.

Testing 89

Testing Strategy Modeling ROI 89

CIO Business Intelligence

OCTOBER 23, 2024

Sonnet large language model has gained a new ability: operating a computer. The new ability, which the company is calling “computer use,” is currently in beta test. Up until now, LLM developers have made tools fit the model, producing custom environments where AIs use specially-designed tools to complete various tasks.

Testing 141

Testing Software Risk Modeling 141

CIO Business Intelligence

OCTOBER 15, 2024

But continuous deployment isn’t always appropriate for your business , stakeholders don’t always understand the costs of implementing robust continuous testing , and end-users don’t always tolerate frequent app deployments during peak usage. CrowdStrike recently made the news about a failed deployment impacting 8.5

Enterprise 143

Enterprise Testing Business Objectives Experimentation 143

CIO Business Intelligence

SEPTEMBER 26, 2024

That’s because significant challenges persist in leveraging GenAI’s large language models (LLMs). One is the security and compliance risks inherent to GenAI. To make accurate, data-driven decisions, businesses need to feed LLMs with proprietary information, but this risks exposing sensitive data to unauthorized parties.

Enterprise 130

Enterprise Data-driven Risk Testing 130

O'Reilly on Data

JUNE 15, 2023

The world changed on November 30, 2022 as surely as it did on August 12, 1908 when the first Model T left the Ford assembly line. Should we risk loss of control of our civilization?” The creators of generative AI systems and Large Language Models already have tools for monitoring, modifying, and optimizing them.

Metrics 290

Metrics Reporting Measurement Finance 290

erwin

SEPTEMBER 25, 2020

Understanding the benefits of data modeling is more important than ever. Data modeling is the process of creating a data model to communicate data requirements, documenting data structures and entity types. In this post: What Is a Data Model? Why Is Data Modeling Important? What’s the Best Data Modeling Tool?

Modeling 111

Modeling Cost-Benefit Visualization Data Warehouse 111

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

Rocket-Powered Data Science

FEBRUARY 15, 2023

While generative AI has been around for several years , the arrival of ChatGPT (a conversational AI tool for all business occasions, built and trained from large language models) has been like a brilliant torch brought into a dark room, illuminating many previously unseen opportunities. So, if you have 1 trillion data points (g.,

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290