O'Reilly on Data

MARCH 20, 2019

Apply fair and private models, white-hat and forensic model debugging, and common sense to protect machine learning models from malicious actors. Like many others, I’ve known for some time that machine learning models themselves could pose security risks.

Modeling 272

Modeling Machine Learning Predictive Modeling Consulting 272

Analytics Vidhya

JANUARY 15, 2024

In a startling revelation, researchers at Anthropic have uncovered a disconcerting aspect of Large Language Models (LLMs) – their capacity to behave deceptively in specific situations, eluding conventional safety measures.

Measurement 315

Measurement Risk Modeling Analytics 315

datapine

MAY 31, 2022

More and more CRM, marketing, and finance-related tools use SaaS business intelligence and technology, and even Adobe’s Creative Suite has adopted the model. This increases the risks that can arise during the implementation or management process. The next part of our cloud computing risks list involves costs.

Risk 237

Risk Cost-Benefit Business Intelligence Data-driven 237

CIO Business Intelligence

MARCH 25, 2025

CIOs perennially deal with technical debts risks, costs, and complexities. While the impacts of legacy systems can be quantified, technical debt is also often embedded in subtler ways across the IT ecosystem, making it hard to account for the full list of issues and risks.

Risk 123

Risk Cost-Benefit Data-driven Digital Transformation 123

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. This creates new risks around data privacy, security, and consistency, making it harder for CIOs to maintain control.

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

CIO Business Intelligence

JANUARY 23, 2024

Regardless of where organizations are in their digital transformation, CIOs must provide their board of directors, executive committees, and employees definitions of successful outcomes and measurable key performance indicators (KPIs). He suggests, “Choose what you measure carefully to achieve the desired results.

Measurement 141

Measurement Digital Transformation KPI Metrics 141

CIO Business Intelligence

OCTOBER 22, 2024

By 2028, 40% of large enterprises will deploy AI to manipulate and measure employee mood and behaviors, all in the name of profit. “AI By 2027, 70% of healthcare providers will include emotional-AI-related terms and conditions in technology contracts or risk billions in financial harm.

Enterprise 135

Enterprise Risk IoT Cost-Benefit 135

O'Reilly on Data

JULY 28, 2020

These measures are commonly referred to as guardrail metrics , and they ensure that the product analytics aren’t giving decision-makers the wrong signal about what’s actually important to the business. When a measure becomes a target, it ceases to be a good measure ( Goodhart’s Law ). Any metric can and will be abused.

Marketing 364

Marketing Experimentation Metrics Testing 364

CIO Business Intelligence

DECEMBER 20, 2024

As concerns about AI security, risk, and compliance continue to escalate, practical solutions remain elusive. as AI adoption and risk increases, its time to understand why sweating the small and not-so-small stuff matters and where we go from here. isnt intentionally or accidentally exfiltrated into a public LLM model?

Risk 105

Risk Measurement Modeling Interactive 105

CIO Business Intelligence

DECEMBER 9, 2024

This is particularly true with enterprise deployments as the capabilities of existing models, coupled with the complexities of many business workflows, led to slower progress than many expected. Assuming a technology can capture these risks will fail like many knowledge management solutions did in the 90s by trying to achieve the impossible.

Enterprise 116

Enterprise ROI Cost-Benefit Measurement 116

O'Reilly on Data

JULY 15, 2019

In recent posts, we described requisite foundational technologies needed to sustain machine learning practices within organizations, and specialized tools for model development, model governance, and model operations/testing/monitoring. Note that the emphasis of SR 11-7 is on risk management.). Sources of model risk.

Machine Learning 48

Machine Learning Management Enterprise Risk Management 48

CIO Business Intelligence

JANUARY 21, 2025

Deloittes State of Generative AI in the Enterprise reports nearly 70% have moved 30% or fewer of their gen AI experiments into production, and 41% of organizations have struggled to define and measure the impacts of their gen AI efforts.

Cost-Benefit 142

Cost-Benefit Data-driven Strategy Marketing 142

O'Reilly on Data

OCTOBER 22, 2020

Taking the time to work this out is like building a mathematical model: if you understand what a company truly does, you don’t just get a better understanding of the present, but you can also predict the future. Since I work in the AI space, people sometimes have a preconceived notion that I’ll only talk about data and models.

Consulting 358

Consulting Risk Cost-Benefit Modeling 358

CIO Business Intelligence

AUGUST 21, 2024

According to Gartner, an agent doesn’t have to be an AI model. Starting in 2018, the agency used agents, in the form of Raspberry PI computers running biologically-inspired neural networks and time series models, as the foundation of a cooperative network of sensors. “It They also had extreme measurement sensitivity.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

DECEMBER 10, 2024

Set clear, measurable metrics around what you want to improve with generative AI, including the pain points and the opportunities, says Shaown Nandi, director of technology at AWS. In HR, measure time-to-hire and candidate quality to ensure AI-driven recruitment aligns with business goals.

Sales 143

Sales Metrics Data-driven Unstructured Data 143

O'Reilly on Data

MAY 27, 2020

Using the new scores, Apgar and her colleagues proved that many infants who initially seemed lifeless could be revived, with success or failure in each case measured by the difference between an Apgar score at one minute after birth, and a second score taken at five minutes. Credit scores.

Risk 361

Risk Testing Reporting Measurement 361

CIO Business Intelligence

AUGUST 13, 2024

Integrating AI and large language models (LLMs) into business operations unlocks new possibilities for innovation and efficiency, offering the opportunity to grow your top line revenue, and improve bottom line profitability. How can you close security gaps related to the surge in AI apps in order to balance both the benefits and risks of AI?

Risk 119

Risk Cost-Benefit Enterprise Modeling 119

O'Reilly on Data

SEPTEMBER 15, 2020

In my book, I introduce the Technical Maturity Model: I define technical maturity as a combination of three factors at a given point of time. Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., Technical competence results in reduced risk and uncertainty.

Advertising 357

Advertising Cost-Benefit ROI Machine Learning 357

CIO Business Intelligence

MARCH 4, 2025

From AI models that boost sales to robots that slash production costs, advanced technologies are transforming both top-line growth and bottom-line efficiency. The takeaway is clear: embrace deep tech now, or risk being left behind by those who do. Today, that timeline is shrinking dramatically. Thats a remarkably short horizon for ROI.

Technology 118

Technology ROI Cost-Benefit Manufacturing 118

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data.

Management 363

Management Machine Learning Experimentation Metrics 363

CIO Business Intelligence

JANUARY 21, 2025

As a secondary measure, we are now evaluating a few deepfake detection tools that can be integrated into our business productivity apps, in particular for Zoom or Teams, to continuously detect deepfakes. Data poisoning and model manipulation are emerging as serious concerns for those of us in cybersecurity.

Behavioral Analytics 122

Behavioral Analytics Data-driven Risk Insurance 122

CIO Business Intelligence

JANUARY 3, 2024

According to an O’Reilly survey released late last month, 23% of companies are using one of OpenAI’s models. Other respondents said they aren’t using any generative AI models, are building their own, or are using an open-source alternative. And it’s not just start-ups that can expose an enterprise to AI-related third-party risk.

Risk 131

Risk Modeling Enterprise Marketing 131

CIO Business Intelligence

OCTOBER 9, 2024

However, this enthusiasm may be tempered by a host of challenges and risks stemming from scaling GenAI. Depending on your needs, large language models (LLMs) may not be necessary for your operations, since they are trained on massive amounts of text and are largely for general use.

Strategy 143

Strategy Modeling Management Data Lake 143

CIO Business Intelligence

NOVEMBER 13, 2024

One is going through the big areas where we have operational services and look at every process to be optimized using artificial intelligence and large language models. But a substantial 23% of respondents say the AI has underperformed expectations as models can prove to be unreliable and projects fail to scale.

IT 141

IT Consulting ROI Cost-Benefit 141

CIO Business Intelligence

JANUARY 21, 2025

Excessive infrastructure costs: About 21% of IT executives point to the high cost of training models or running GenAI apps as a major concern. These concerns emphasize the need to carefully balance the costs of GenAI against its potential benefits, a challenge closely tied to measuring ROI. million in 2025 to $7.45

Cost-Benefit 98

Cost-Benefit ROI Measurement Strategy 98

O'Reilly on Data

MAY 14, 2020

Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model. In reality, many candidate models (frequently hundreds or even thousands) are created during the development process. Modelling: The model is often misconstrued as the most important component of an AI product.

Management 364

Management Experimentation B2B Machine Learning 364

CIO Business Intelligence

NOVEMBER 14, 2024

Online will become increasingly central, with the launch of new collections and models, as well as opening in new markets, transacting in different currencies, and using in-depth analytics to make quick decisions.” BPS also adopts proactive thinking, a risk-based framework for strategic alignment and compliance with business objectives.

ROI 119

ROI B2B Business Objectives Digital Transformation 119

CIO Business Intelligence

MARCH 31, 2025

The 2024 Security Priorities study shows that for 72% of IT and security decision makers, their roles have expanded to accommodate new challenges, with Risk management, Securing AI-enabled technology and emerging technologies being added to their plate. Regular engagement with the board and business leaders ensures risk visibility.

Data-driven 71

Data-driven Risk Risk Management Metrics 71

CIO Business Intelligence

MARCH 5, 2025

If they decide a project could solve a big enough problem to merit certain risks, they then make sure they understand what type of data will be needed to address the solution. The next thing is to make sure they have an objective way of testing the outcome and measuring success. But we dont ignore the smaller players.

Testing 82

Testing Measurement Technology Experimentation 82

CIO Business Intelligence

JANUARY 30, 2025

These changes can expose businesses to risks and vulnerabilities such as security breaches, data privacy issues and harm to the companys reputation. It also includes managing the risks, quality and accountability of AI systems and their outcomes. It is easy to see how the detractions can get in the way. AI governance.

Testing 93

Testing Metrics Cost-Benefit Modeling 93

O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, The GPT-series LLMs are also called “foundation models.” GPT-2, 3, 3.5,

IT 344

IT Modeling Testing Risk 344

CIO Business Intelligence

JANUARY 30, 2025

The aim is to provide a framework that encourages early implementation of some of the measures in the act and to encourage organizations to make public the practices and processes they are implementing to achieve compliance even before the statutory deadline.In

Enterprise 128

Enterprise Risk Reporting Measurement 128

CIO Business Intelligence

NOVEMBER 21, 2024

It’s very easy to get quick success with a prototype, but there is hidden cost involved in making your data AI ready, training your AI models with corporate data, tuning it post deployment, putting the controls to limit abuse, biases, and hallucinations.” The cost “just compounds exponentially,” he adds. “It

Management 124

Management Interactive ROI Software 124

Smart Data Collective

FEBRUARY 14, 2023

However, it is important to understand the benefits and risks associated with cloud computing before making the commitment. However, there are some risks associated with using cloud-based software for business purposes. Firstly, there is always the risk of data breaches due to cyber-attacks or human error.

Risk 119

Risk Cost-Benefit Software Technology 119

CIO Business Intelligence

DECEMBER 28, 2023

The need to manage risk, adhere to regulations, and establish processes to govern those tasks has been part of running an organization as long as there have been businesses to run. Furthermore, the State of Risk & Compliance Report, from GRC software maker NAVEX, found that 20% described their programs as early stage. What is GRC?

Risk 115

Risk Risk Management Strategy Enterprise 115

Rocket-Powered Data Science

FEBRUARY 15, 2023

While generative AI has been around for several years , the arrival of ChatGPT (a conversational AI tool for all business occasions, built and trained from large language models) has been like a brilliant torch brought into a dark room, illuminating many previously unseen opportunities.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

O'Reilly on Data

APRIL 23, 2024

Some of those innovations, like Amazon’s cloud computing business, represented enormous new markets and a new business model. Google, for example, invented the Large Language model architecture that underlies today’s disruptive AI startups. These companies did continue to innovate. I think not.

Advertising 338

Advertising Cost-Benefit Marketing Risk 338