O'Reilly on Data

MARCH 23, 2023

What is it, how does it work, what can it do, and what are the risks of using it? ChatGPT, or something built on ChatGPT, or something that’s like ChatGPT, has been in the news almost constantly since ChatGPT was opened to the public in November 2022. A quick scan of the web will show you lots of things that ChatGPT can do. It’s much more.

IT 344

IT Modeling Testing Risk 344

O'Reilly on Data

MARCH 25, 2025

Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. What breaks your app in production isnt always what you tested for in dev! The way out?

Testing 174

Testing Data-driven Software Measurement 174

CIO Business Intelligence

NOVEMBER 4, 2024

This may involve embracing redundancies or testing new tools for future operations. Organizations can maintain high-risk parts of their legacy VMware infrastructure while exploring how an alternative hypervisor can run business-critical applications and build new capabilities,” said Carter. It took about 18 months. You need to plan.

IT 125

IT Forecasting Strategy Testing 125

CIO Business Intelligence

FEBRUARY 26, 2025

All of this creates new challenges, on top of those already posed by the gen AI itself. Plus, unlike traditional automations, agentic systems are non-deterministic. This puts them at odds with legacy platforms, which are universally very deterministic. If you want to strike oil, you have to drill through the granite to get to it.

IT 141

IT Enterprise Interactive Data Quality 141

CIO Business Intelligence

MARCH 25, 2025

CIOs perennially deal with technical debts risks, costs, and complexities. While the impacts of legacy systems can be quantified, technical debt is also often embedded in subtler ways across the IT ecosystem, making it hard to account for the full list of issues and risks.

Risk 123

Risk Cost-Benefit Data-driven Digital Transformation 123

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

CIO Business Intelligence

FEBRUARY 20, 2025

However, this perception of resilience must be backed up by robust, tested strategies that can withstand real-world threats. Given the rapid evolution of cyber threats and continuous changes in corporate IT environments, failing to update and test resilience plans can leave businesses exposed when attacks or major outages occur.

IT 99

IT Testing Strategy Data Processing 99

CIO Business Intelligence

MARCH 19, 2025

There are risks around hallucinations and bias, says Arnab Chakraborty, chief responsible AI officer at Accenture. Meanwhile, in December, OpenAIs new O3 model, an agentic model not yet available to the public, scored 72% on the same test. The next evolution of AI has arrived, and its agentic.

Software 143

Software Risk Enterprise Cost-Benefit 143

DataKitchen

MARCH 20, 2025

Rather than concentrating on individual tables, these teams devote their resources to ensuring each pipeline, workflow, or DAG (Directed Acyclic Graph) is transparent, thoroughly tested, and easily deployable through automation. Their data tables become dependable by-products of meticulously crafted and managed workflows.

Data Quality 164

Data Quality Testing Metrics Management 164

CIO Business Intelligence

JULY 25, 2024

It also highlights the downsides of concentration risk. What is concentration risk? Looking to the future, IT leaders must bring stronger focus on “concentration risk”and how these supply chain risks can be better managed. In layman’s terms, it simply means putting all your eggs in one basket.

Risk 144

Risk Cost-Benefit Strategy Enterprise 144

CIO Business Intelligence

APRIL 23, 2024

As CIO, you’re in the risk business. Or rather, every part of your responsibilities entails risk, whether you’re paying attention to it or not. There are, for example, those in leadership roles who, while promoting the value of risk-taking, also insist on “holding people accountable.” You can’t lose.

Risk 116

Risk IT Insurance Risk Management 116

O'Reilly on Data

FEBRUARY 26, 2021

These articles show you how to minimize your risk at every stage of the project, from initial planning through to post-deployment monitoring and testing. A couple of years ago, Pete Skomoroch, Roger Magoulas, and I talked about the problems of being a product manager for an AI product. That’s true at every stage of the process.

Management 349

Management Testing Data Science Risk 349

O'Reilly on Data

MAY 27, 2020

Algorithms tell stories about who people are. The first story an algorithm told about me was that my life was in danger. It was 7:53 pm on a clear Monday evening in September of 1981, at the Columbia Hospital for Women in Washington DC. I was exactly one minute old. You get two points for waving your arms and legs, for instance.)

Risk 361

Risk Testing Reporting Measurement 361

O'Reilly on Data

OCTOBER 22, 2020

“ On peut interroger n’importe qui, dans n’importe quel état; ce sont rarement les réponses qui apportent la vérité, mais l’enchaînement des questions. “ “ You can interrogate anyone, no matter what their state of being. “ – Inspector Pastor in La Fée Carabine, by Daniel Pennac. And that’s fine.

Consulting 358

Consulting Risk Cost-Benefit Modeling 358

CIO Business Intelligence

MAY 24, 2024

GRC certifications validate the skills, knowledge, and abilities IT professionals have to manage governance, risk, and compliance (GRC) in the enterprise. Enter the need for competent governance, risk and compliance (GRC) professionals. Why are GRC certifications important? Is GRC certification worth it?

Risk 120

Risk Risk Management Management Enterprise 120

O'Reilly on Data

JULY 28, 2020

Product Managers are responsible for the successful development, testing, release, and adoption of a product, and for leading the team that implements those milestones. The Core Responsibilities of the AI Product Manager. Product managers for AI must satisfy these same responsibilities, tuned for the AI lifecycle. Identifying the problem.

Marketing 364

Marketing Experimentation Metrics Testing 364

O'Reilly on Data

NOVEMBER 28, 2023

Unexpected outcomes, security, safety, fairness and bias, and privacy are the biggest risks for which adopters are testing. We’re not encouraging skepticism or fear, but companies should start AI products with a clear understanding of the risks, especially those risks that are specific to AI. What’s Holding AI Back?

Enterprise 342

Enterprise Testing Modeling Reporting 342

datapine

JANUARY 24, 2021

In our cutthroat digital age, the importance of setting the right data analysis questions can define the overall success of a business. That being said, it seems like we’re in the midst of a data analysis crisis. That being said, it seems like we’re in the midst of a data analysis crisis. Data Is Only As Good As The Questions You Ask.

IT 317

IT Statistics KPI Data-driven 317

CIO Business Intelligence

AUGUST 21, 2024

Adding smarter AI also adds risk, of course. “At The big risk is you take the humans out of the loop when you let these into the wild.” According to Gartner, an agent doesn’t have to be an AI model. It can also be a software program or another computational entity — or a robot. And, yes, enterprises are already deploying them.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

JANUARY 30, 2025

These changes can expose businesses to risks and vulnerabilities such as security breaches, data privacy issues and harm to the companys reputation. Artificial Intelligence (AI) technologies are moving faster than previous technologies and it is transforming companies and industries at an extraordinary rate.

Testing 93

Testing Metrics Cost-Benefit Modeling 93

CIO Business Intelligence

MARCH 4, 2025

The takeaway is clear: embrace deep tech now, or risk being left behind by those who do. No wonder nearly every CEO is talking about AI: those who lag in AI adoption risk falling behind competitors capabilities. Today, that timeline is shrinking dramatically. It was hard to imagine this pace 5-10 years ago.

Technology 118

Technology ROI Cost-Benefit Manufacturing 118

CIO Business Intelligence

OCTOBER 17, 2024

While tech debt refers to shortcuts taken in implementation that need to be addressed later, digital addiction results in the accumulation of poorly vetted, misused, or unnecessary technologies that generate costs and risks. million machines worldwide, serves as a stark reminder of these risks.

Risk 108

Risk Experimentation Testing Strategy 108

O'Reilly on Data

DECEMBER 12, 2019

1] This includes C-suite executives, front-line data scientists, and risk, legal, and compliance personnel. These recommendations are based on our experience, both as a data scientist and as a lawyer, focused on managing the risks of deploying ML. Not least is the broadening realization that ML models can fail. Sensitivity analysis.

Machine Learning 275

Machine Learning Modeling Testing Risk Management 275

CIO Business Intelligence

JANUARY 21, 2025

DevOps teams follow their own practices of using continuous integration and continuous deployment (CI/CD) tools to automatically merge code changes and automate testing steps to deploy changes more frequently and reliably. Agentic AI promises to transform enterprise IT work.

Risk 98

Risk Data-driven Interactive Management 98

CIO Business Intelligence

JUNE 13, 2024

Not instant perfection The NIPRGPT experiment is an opportunity to conduct real-world testing, measuring generative AI’s computational efficiency, resource utilization, and security compliance to understand its practical applications. For now, AFRL is experimenting with self-hosted open-source LLMs in a controlled environment.

Testing 120

Testing Experimentation Data Processing Modeling 120

CIO Business Intelligence

JULY 25, 2024

IT managers are often responsible for not just overseeing an organization’s IT infrastructure but its IT teams as well. To succeed, you need to understand the fundamentals of security, data storage, hardware, software, networking, and IT management frameworks — and how they all work together to deliver business value.

IT 131

IT Management Risk Risk Management 131

AWS Big Data

JANUARY 9, 2025

This simplifies data modification processes, which is crucial for ingesting and updating large volumes of market and trade data, quickly iterating on backtesting and reprocessing workflows, and maintaining detailed audit trails for risk and compliance requirements. Business impact heavily relies on quality data (garbage in, garbage out).

Metadata 111

Metadata Snapshot Cost-Benefit Optimization 111

CIO Business Intelligence

MARCH 5, 2025

If they decide a project could solve a big enough problem to merit certain risks, they then make sure they understand what type of data will be needed to address the solution. The next thing is to make sure they have an objective way of testing the outcome and measuring success. But we dont ignore the smaller players.

Testing 83

Testing Measurement Technology Experimentation 83

CIO Business Intelligence

MARCH 4, 2025

In fact, successful recovery from cyberattacks and other disasters hinges on an approach that integrates business impact assessments (BIA), business continuity planning (BCP), and disaster recovery planning (DRP) including rigorous testing. See also: How resilient CIOs future-proof to mitigate risks.)

Testing 65

Testing Risk Consulting Risk Management 65

CIO Business Intelligence

SEPTEMBER 6, 2023

The discussions address changing regulatory and compliance requirements, and reveal vulnerabilities and threats for risk mitigation.” Ongoing IT security strategy conversations should address the organization’s cyber risk and arrive at strategic objectives, Albrecht says. Are our systems adequately modernized for security?

IT 142

IT ROI Risk Strategy 142

DataKitchen

NOVEMBER 20, 2021

A DataOps Engineer can make test data available on demand. We have automated testing and a system for exception reporting, where tests identify issues that need to be addressed. The DataOps Engineer leverages a common framework that encompasses the end-to-end data lifecycle. Shepherding Processes Across the Corporate Landscape.

Testing 246

Testing Machine Learning Data Warehouse Analytics 246

CIO Business Intelligence

SEPTEMBER 26, 2023

Management rules typically exist to enable faultless decision-making, set a foundation for consistent operation, and provide protection from risk, observes Ola Chowning, a partner at global technology research and advisory firm ISG. Breaking a rule often happens after the CIO weighs the risk of removing or retaining a mandate,” she notes.

IT 141

IT Risk Cost-Benefit Consulting 141

CIO Business Intelligence

OCTOBER 17, 2024

Allegations of fraud and security risks The indictment details that the fraudulent certification, combined with misleading claims about the facility’s capabilities, led the SEC to award Jain’s company the contract in 2012. The scheme allegedly put the SEC’s data security and operational integrity at risk. million on the contract​.

Broadcasting 122

Broadcasting Risk Reporting Measurement 122

CIO Business Intelligence

OCTOBER 8, 2024

You risk adding to the hype where there will be no observable value. Whatever it is, it will steal everyone’s job. There is no way it will ever be secure. There is a race to be the first to expose your leveraging of it. Nobody knows what it is, what it really does, and you must become an expert in short order. But is this really wise?

Risk 125

Risk Technology Cost-Benefit Internet of Things 125

CIO Business Intelligence

DECEMBER 2, 2024

Business risk (liabilities): “Our legacy systems increase our cybersecurity exposure by 40%.” Don’t get bogged down in testing multiple solutions that never see the light of day. Breaking it down into these categories also shows the impact on the business in a way that every board member will understand.

ROI 137

ROI Cost-Benefit Metrics Strategy 137

CIO Business Intelligence

JANUARY 30, 2025

particular, companies that use AI systems can share their voluntary commitments to transparency and risk control. At least half of the current AI Pact signatories (numbering more than 130) have made additional commitments, such as risk mitigation, human oversight and transparency in generative AI content.

Enterprise 128

Enterprise Risk Reporting Measurement 128

DataKitchen

SEPTEMBER 20, 2021

The best way to ensure error-free execution of data production is through automated testing and monitoring. The DataKitchen Platform enables data teams to integrate testing and observability into data pipeline orchestrations. Automated tests work 24×7 to ensure that the results of each processing stage are accurate and correct.

Testing 246

Testing Metrics Measurement Dashboards 246