O'Reilly on Data

DECEMBER 12, 2019

Not least is the broadening realization that ML models can fail. And that’s why model debugging, the art and science of understanding and fixing problems in ML models, is so critical to the future of ML. Because all ML models make mistakes, everyone who cares about ML should also care about model debugging. [1]

Machine Learning 196

Machine Learning Modeling Testing Risk Management 196

CIO Business Intelligence

OCTOBER 16, 2024

In the executive summary of the updated RSP , Anthropic stated, “in September 2023, we released our Responsible Scaling Policy (RSP), a public commitment not to train or deploy models capable of causing catastrophic harm unless we have implemented safety and security measures that will keep risks below acceptable levels.

Risk 117

Risk IT Measurement Modeling 117

CIO Business Intelligence

SEPTEMBER 17, 2024

CISOs can only know the performance and maturity of their security program by actively measuring it themselves; after all, to measure is to know. However, CISOs aren’t typically measuring their security program proactively or methodically to understand their current security program.

Measurement 111

Measurement Metrics Risk Dashboards 111

CIO Business Intelligence

AUGUST 21, 2024

According to Gartner, an agent doesn’t have to be an AI model. Starting in 2018, the agency used agents, in the form of Raspberry PI computers running biologically-inspired neural networks and time series models, as the foundation of a cooperative network of sensors. “It They also had extreme measurement sensitivity.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

NOVEMBER 7, 2024

The UK government has introduced an AI assurance platform, offering British businesses a centralized resource for guidance on identifying and managing potential risks associated with AI, as part of efforts to build trust in AI systems. Meanwhile, the measures could also introduce fresh challenges for businesses, particularly SMEs.

Risk 130

Risk Management Measurement Consulting 130

CIO Business Intelligence

JANUARY 3, 2024

According to an O’Reilly survey released late last month, 23% of companies are using one of OpenAI’s models. Other respondents said they aren’t using any generative AI models, are building their own, or are using an open-source alternative. And it’s not just start-ups that can expose an enterprise to AI-related third-party risk.

Risk 131

Risk Modeling Enterprise Marketing 131

O'Reilly on Data

JUNE 15, 2023

The world changed on November 30, 2022 as surely as it did on August 12, 1908 when the first Model T left the Ford assembly line. Should we risk loss of control of our civilization?” The creators of generative AI systems and Large Language Models already have tools for monitoring, modifying, and optimizing them.

Metrics 289

Metrics Reporting Measurement Finance 289

O'Reilly on Data

MAY 27, 2020

Using the new scores, Apgar and her colleagues proved that many infants who initially seemed lifeless could be revived, with success or failure in each case measured by the difference between an Apgar score at one minute after birth, and a second score taken at five minutes. Credit scores.

Risk 355

Risk Testing Reporting Measurement 355

CIO Business Intelligence

AUGUST 13, 2024

Integrating AI and large language models (LLMs) into business operations unlocks new possibilities for innovation and efficiency, offering the opportunity to grow your top line revenue, and improve bottom line profitability. How can you close security gaps related to the surge in AI apps in order to balance both the benefits and risks of AI?

Risk 119

Risk Cost-Benefit Enterprise Modeling 119

CIO Business Intelligence

OCTOBER 9, 2024

However, this enthusiasm may be tempered by a host of challenges and risks stemming from scaling GenAI. Depending on your needs, large language models (LLMs) may not be necessary for your operations, since they are trained on massive amounts of text and are largely for general use.

Strategy 143

Strategy Modeling Management Data Lake 143

O'Reilly on Data

OCTOBER 22, 2020

Taking the time to work this out is like building a mathematical model: if you understand what a company truly does, you don’t just get a better understanding of the present, but you can also predict the future. Since I work in the AI space, people sometimes have a preconceived notion that I’ll only talk about data and models.

Consulting 301

Consulting Risk Cost-Benefit Modeling 301

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data.

Management 338

Management Machine Learning Experimentation Metrics 338

O'Reilly on Data

MAY 14, 2020

Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model. In reality, many candidate models (frequently hundreds or even thousands) are created during the development process. Modelling: The model is often misconstrued as the most important component of an AI product.

Management 363

Management Experimentation B2B Machine Learning 363

CIO Business Intelligence

DECEMBER 28, 2023

The need to manage risk, adhere to regulations, and establish processes to govern those tasks has been part of running an organization as long as there have been businesses to run. Furthermore, the State of Risk & Compliance Report, from GRC software maker NAVEX, found that 20% described their programs as early stage. What is GRC?

Risk 115

Risk Risk Management Strategy Enterprise 115

O'Reilly on Data

SEPTEMBER 15, 2020

In my book, I introduce the Technical Maturity Model: I define technical maturity as a combination of three factors at a given point of time. Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., Technical competence results in reduced risk and uncertainty.

Advertising 299

Advertising Cost-Benefit ROI Machine Learning 299

CIO Business Intelligence

MARCH 25, 2024

Modern digital organisations tend to use an agile approach to delivery, with cross-functional teams, product-based operating models , and persistent funding. But to deliver transformative initiatives, CIOs need to embrace the agile, product-based approach, and that means convincing the CFO to switch to a persistent funding model.

Modeling 124

Modeling Cost-Benefit IT ROI 124

Rocket-Powered Data Science

FEBRUARY 15, 2023

While generative AI has been around for several years , the arrival of ChatGPT (a conversational AI tool for all business occasions, built and trained from large language models) has been like a brilliant torch brought into a dark room, illuminating many previously unseen opportunities.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

CIO Business Intelligence

OCTOBER 24, 2023

Cloud applications, in essence, have become organizations’ crown jewels and developers are measured on how quickly they can build and deploy them. In light of this, developer teams are beginning to turn to AI-enabled tools like large language models (LLMs) to simplify and automate tasks.

Modeling 119

Modeling Risk Reporting Machine Learning 119

Smart Data Collective

FEBRUARY 14, 2023

However, it is important to understand the benefits and risks associated with cloud computing before making the commitment. However, there are some risks associated with using cloud-based software for business purposes. Firstly, there is always the risk of data breaches due to cyber-attacks or human error.

Risk 112

Risk Cost-Benefit Software Technology 112

CIO Business Intelligence

OCTOBER 10, 2022

Bass further expanded the concept to include ways for measuring the success of transformational leadership. This model encourages leaders to demonstrate authentic, strong leadership with the idea that employees will be inspired to follow suit. Transformational leadership model. This leadership model doesn’t try to innovate.

Modeling 145

Modeling Digital Transformation Software Manufacturing 145

DataKitchen

NOVEMBER 27, 2024

For example, metrics like the percentage of missing values help measure completeness, while deviations from authoritative sources gauge accuracy. For instance, a bank might use a CDE dashboard to monitor the accuracy of customer credit scores, which are vital for regulatory compliance and effective risk modeling.

Data Quality 219

Data Quality Dashboards Metrics Cost-Benefit 219

David Menninger's Analyst Perspectives

OCTOBER 9, 2024

It helps build trust in the results of AI models, it helps ensure compliance with regulations and it is necessary to meet internal governance requirements. Effective AI governance must encompass various dimensions, including data privacy, model drift, hallucinations, toxicity and perhaps most importantly, bias.

Metrics 147

Metrics Measurement Machine Learning Modeling 147

DataKitchen

SEPTEMBER 20, 2021

For example, data cleansing, ETL, running a model, or even provisioning cloud infrastructure. Minimize Deployment Risk. Measurement DataOps. Once you’ve made progress with your production and development processes, it’s time to start measuring and improving your processes with Measurement DataOps.

Testing 246

Testing Metrics Measurement Dashboards 246

CIO Business Intelligence

OCTOBER 24, 2024

Take for instance large language models (LLMs) for GenAI. From prompt injections to poisoning training data, these critical vulnerabilities are ripe for exploitation, potentially leading to increased security risks for businesses deploying GenAI. This puts businesses at greater risk for data breaches.

Risk 128

Risk Measurement Optimization Machine Learning 128

O'Reilly on Data

JULY 20, 2021

When we’re building shared devices with a user model, that model quickly runs into limitations. That model doesn’t fit reality: the identity of a communal device isn’t a single person, but everyone who can interact with it. When we consider the risk associated with an action, we need to understand its privacy implications.

Risk 306

Risk Software Interactive Modeling 306

CIO Business Intelligence

SEPTEMBER 10, 2024

The US Department of Commerce’s Bureau of Industry and Security (BIS) plans to introduce mandatory reporting requirements for developers of advanced AI models and cloud computing providers.

Reporting 131

Reporting Measurement Testing Risk 131

IBM Big Data Hub

SEPTEMBER 7, 2023

Today we are announcing our latest addition: a new family of IBM-built foundation models which will be available in watsonx.ai , our studio for generative AI, foundation models and machine learning. Collectively named “Granite,” these multi-size foundation models apply generative AI to both language and code.

Modeling 101

Modeling Risk Unstructured Data Enterprise 101

O'Reilly on Data

JUNE 18, 2024

Last month, TheNew York Times claimed that tech giants OpenAI and Google have waded into a copyright gray area by transcribing the vast volume of YouTube videos and using that text as additional training data for their AI models despite terms of service that prohibit such efforts and copyright law that the Times argues places them in dispute.

Advertising 353

Advertising Modeling Publishing Data Processing 353

CIO Business Intelligence

AUGUST 9, 2024

In cases where privacy is essential, we try to anonymize as much as possible and then move on to training the model,” says University of Florence technologist Vincenzo Laveglia. “A We have to make sure there’s a balance between various classes of information, otherwise the model becomes an expert on one topic and very uncertain on others.”

Risk 138

Risk Measurement Modeling Data Lake 138

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, The GPT-series LLMs are also called “foundation models.” GPT-2, 3, 3.5,

IT 267

IT Modeling Testing Risk 267

O'Reilly on Data

APRIL 23, 2024

Some of those innovations, like Amazon’s cloud computing business, represented enormous new markets and a new business model. Google, for example, invented the Large Language model architecture that underlies today’s disruptive AI startups. These companies did continue to innovate. I think not.

Advertising 276

Advertising Cost-Benefit Marketing Risk 276

datapine

OCTOBER 7, 2020

What gets measured gets done.” – Peter Drucker. By setting operational performance measures, you will know what is happening at every stage of your business. Since every business is different, it is essential to establish specific metrics and KPIs to measure, follow, calculate, and evaluate. Who will measure it?

KPI 269

KPI Metrics Sales Visualization 269

CIO Business Intelligence

MAY 22, 2024

The signatories agreed to publish — if they have not done so already — safety frameworks outlining on how they will measure the risks of their respective AI models. The risks might include the potential for misuse of the model by a bad actor, for instance.

Risk Management 122

Risk Management Risk Publishing Modeling 122

CIO Business Intelligence

JUNE 27, 2024

The issue has become a concern for builders of generative AI models and the enterprises that use them, as some data sets used in AI training have legally and ethically uncertain origins. Trade associations like the DPA may play a role in supporting the enforcement of such legislation and advocating for other similar measures.

Risk 128

Risk Measurement Risk Management Strategy 128

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. This creates new risks around data privacy, security, and consistency, making it harder for CIOs to maintain control.

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

datapine

SEPTEMBER 29, 2022

5) How Do You Measure Data Quality? In this article, we will detail everything which is at stake when we talk about DQM: why it is essential, how to measure data quality, the pillars of good quality management, and some data quality control techniques. These needs are then quantified into data models for acquisition and delivery.

Data Quality 241

Data Quality Metrics Data-driven Management 241

CIO Business Intelligence

OCTOBER 18, 2023

The procedure, often called kidney dialysis, cleansing a patient’s blood, substituting for the function of the kidneys, and is not without risk, however. Fresenius’s machine learning model uses electronic health records comprising intradialytic blood pressure measurements and multiple treatment- and patient-level variables.

Predictive Analytics 133

Predictive Analytics Analytics Machine Learning Internet of Things 133