O'Reilly on Data

MARCH 25, 2025

Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. What breaks your app in production isnt always what you tested for in dev! The way out?

Testing 168

Testing Data-driven Software Measurement 168

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

CIO Business Intelligence

JANUARY 7, 2025

Regardless of the driver of transformation, your companys culture, leadership, and operating practices must continuously improve to meet the demands of a globally competitive, faster-paced, and technology-enabled world with increasing security and other operational risks.

Digital Transformation 143

Digital Transformation Experimentation Cost-Benefit Strategy 143

CIO Business Intelligence

MARCH 25, 2025

CIOs perennially deal with technical debts risks, costs, and complexities. While the impacts of legacy systems can be quantified, technical debt is also often embedded in subtler ways across the IT ecosystem, making it hard to account for the full list of issues and risks.

Risk 140

Risk Cost-Benefit Data-driven Digital Transformation 140

CIO Business Intelligence

JANUARY 14, 2025

By articulating fitness functions automated tests tied to specific quality attributes like reliability, security or performance teams can visualize and measure system qualities that align with business goals. Technical foundation Conversation starter : Are we maintaining reliable roads and utilities, or are we risking gridlock?

Enterprise 119

Enterprise Technology Metrics Measurement 119

AWS Big Data

JANUARY 9, 2025

In our previous post Backtesting index rebalancing arbitrage with Amazon EMR and Apache Iceberg , we showed how to use Apache Iceberg in the context of strategy backtesting. This capability is particularly valuable in maintaining the integrity of backtests and the reliability of trading strategies.

Metadata 106

Metadata Snapshot Cost-Benefit Optimization 106

CIO Business Intelligence

DECEMBER 18, 2024

Financial institutions have an unprecedented opportunity to leverage AI/GenAI to expand services, drive massive productivity gains, mitigate risks, and reduce costs. GenAI is also helping to improve risk assessment via predictive analytics.

Forecasting 105

Forecasting Predictive Analytics Risk Finance 105

CIO Business Intelligence

JANUARY 6, 2025

One of them is Katherine Wetmur, CIO for cyber, data, risk, and resilience at Morgan Stanley. Wetmur says Morgan Stanley has been using modern data science, AI, and machine learning for years to analyze data and activity, pinpoint risks, and initiate mitigation, noting that teams at the firm have earned patents in this space.

IT 140

IT Cost-Benefit Measurement Experimentation 140

Smart Data Collective

MAY 7, 2023

million —and organizations are constantly at risk of cyber-attacks and malicious actors. In order to protect your business from these threats, it’s essential to understand what digital transformation entails and how you can safeguard your company from cyber risks. What is cyber risk?

Digital Transformation 129

Digital Transformation Risk Strategy Testing 129

CIO Business Intelligence

APRIL 8, 2025

Fragmented systems, inconsistent definitions, legacy infrastructure and manual workarounds introduce critical risks. I aim to outline pragmatic strategies to elevate data quality into an enterprise-wide capability. This challenge remains deceptively overlooked despite its profound impact on strategy and execution.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

CIO Business Intelligence

MAY 24, 2024

GRC certifications validate the skills, knowledge, and abilities IT professionals have to manage governance, risk, and compliance (GRC) in the enterprise. Enter the need for competent governance, risk and compliance (GRC) professionals. What are GRC certifications? Why are GRC certifications important?

Risk 119

Risk Risk Management Management Enterprise 119

Smart Data Collective

DECEMBER 26, 2022

Let’s get started with a comprehensive cybersecurity strategy for your small business. The first step of a well-planned cybersecurity strategy is identifying the avenues of attack in your system. Before prioritizing your threats, risks, and remedies, determine the rules and regulations that your company is obliged to follow.

Strategy 137

Strategy Testing Risk Software 137

Smart Data Collective

MARCH 23, 2023

Prebuilt features and templates will have already been performance tested, and they typically come at much lower price points than developing a product from scratch. Automate Your Testing AI technology can also help create other AI applications. One of the benefits is that it can help with automating coding and testing.

Software 142

Software Strategy Cost-Benefit Testing 142

CIO Business Intelligence

SEPTEMBER 9, 2024

As IT landscapes and software delivery processes evolve, the risk of inadvertently creating new vulnerabilities increases. These risks are particularly critical for financial services institutions, which are now under greater scrutiny with the Digital Operational Resilience Act ( DORA ).

Risk Management 115

Risk Management Risk Management IT 115

O'Reilly on Data

DECEMBER 12, 2019

1] This includes C-suite executives, front-line data scientists, and risk, legal, and compliance personnel. These recommendations are based on our experience, both as a data scientist and as a lawyer, focused on managing the risks of deploying ML. 6] Debugging may focus on a variety of failure modes (i.e., Sensitivity analysis.

Machine Learning 276

Machine Learning Modeling Testing Risk Management 276

CIO Business Intelligence

AUGUST 21, 2024

Adding smarter AI also adds risk, of course. “At The big risk is you take the humans out of the loop when you let these into the wild.” When it comes to security, though, agentic AI is a double-edged sword with too many risks to count, he says. “We That means the projects are evaluated for the amount of risk they involve.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

OCTOBER 17, 2024

While tech debt refers to shortcuts taken in implementation that need to be addressed later, digital addiction results in the accumulation of poorly vetted, misused, or unnecessary technologies that generate costs and risks. million machines worldwide, serves as a stark reminder of these risks.

Risk 108

Risk Experimentation Testing Strategy 108

CIO Business Intelligence

OCTOBER 24, 2024

Jayesh Chaurasia, analyst, and Sudha Maheshwari, VP and research director, wrote in a blog post that businesses were drawn to AI implementations via the allure of quick wins and immediate ROI, but that led many to overlook the need for a comprehensive, long-term business strategy and effective data management practices.

ROI 127

ROI Data-driven Enterprise Experimentation 127

CIO Business Intelligence

NOVEMBER 14, 2023

Although some continue to leap without looking into cloud deals, the value of developing a comprehensive cloud strategy has become evident. Without a clear cloud strategy and broad leadership support, even value-adding cloud investments may be at risk. There are other risks, too. Why are we really going to cloud?

Strategy 145

Strategy Cost-Benefit IT Key Performance Indicator 145

CIO Business Intelligence

DECEMBER 2, 2024

Business risk (liabilities): “Our legacy systems increase our cybersecurity exposure by 40%.” Suboptimal integration strategies are partly to blame, and on top of this, companies often don’t have security architecture that can handle both people and AI agents working on IT systems. Also, beware the proof-of-concept trap.

ROI 136

ROI Cost-Benefit Metrics Strategy 136

CIO Business Intelligence

NOVEMBER 4, 2024

Rather than wait for a storm to hit, IT professionals map out options and build strategies to ensure business continuity. This may involve embracing redundancies or testing new tools for future operations. The disruption from VMware’s acquisition has led many to reconsider their virtualization strategies and explore new options. “By

IT 124

IT Forecasting Strategy Testing 124

CIO Business Intelligence

MARCH 9, 2022

Today’s cloud strategies revolve around two distinct poles: the “lift and shift” approach, in which applications and associated data are moved to the cloud without being redesigned; and the “cloud-first” approach, in which applications are developed or redesigned specifically for the cloud.

Strategy 145

Strategy Consulting Risk Testing 145

O'Reilly on Data

NOVEMBER 28, 2023

Unexpected outcomes, security, safety, fairness and bias, and privacy are the biggest risks for which adopters are testing. We’re not encouraging skepticism or fear, but companies should start AI products with a clear understanding of the risks, especially those risks that are specific to AI.

Enterprise 339

Enterprise Testing Modeling Reporting 339

CIO Business Intelligence

APRIL 23, 2024

As CIO, you’re in the risk business. Or rather, every part of your responsibilities entails risk, whether you’re paying attention to it or not. There are, for example, those in leadership roles who, while promoting the value of risk-taking, also insist on “holding people accountable.” You can’t lose.

Risk 116

Risk IT Insurance Risk Management 116

Smart Data Collective

DECEMBER 6, 2022

One of the best ways that cybersecurity professionals are leveraging AI is by utilizing SAST strategies. A big part of what enables this constant deployment of new applications is a testing process known as static application security testing, or SAST. It is frequently referred to as “white box testing.”

Strategy 126

Strategy Testing Software Risk 126

CIO Business Intelligence

FEBRUARY 20, 2025

However, this perception of resilience must be backed up by robust, tested strategies that can withstand real-world threats. One major gap in the findings is that four in ten respondents admitted their organization had not reviewed its cyber resilience strategy in the last six months.

IT 98

IT Testing Strategy Data Processing 98

CIO Business Intelligence

JUNE 13, 2024

Not instant perfection The NIPRGPT experiment is an opportunity to conduct real-world testing, measuring generative AI’s computational efficiency, resource utilization, and security compliance to understand its practical applications. For now, AFRL is experimenting with self-hosted open-source LLMs in a controlled environment.

Testing 119

Testing Experimentation Data Processing Modeling 119

CIO Business Intelligence

JANUARY 30, 2025

These changes can expose businesses to risks and vulnerabilities such as security breaches, data privacy issues and harm to the companys reputation. It also includes managing the risks, quality and accountability of AI systems and their outcomes. AI governance is critical and should never be just a regulatory requirement.

Testing 93

Testing Metrics Cost-Benefit Modeling 93

CIO Business Intelligence

MARCH 4, 2025

In fact, successful recovery from cyberattacks and other disasters hinges on an approach that integrates business impact assessments (BIA), business continuity planning (BCP), and disaster recovery planning (DRP) including rigorous testing. See also: How resilient CIOs future-proof to mitigate risks.)

Testing 64

Testing Risk Consulting Risk Management 64

datapine

APRIL 15, 2020

When it comes to implementing and managing a successful BI strategy we have always proclaimed: start small, use the right BI tools , and involve your team. Your Chance: Want to test an agile business intelligence solution? You need to determine if you are going with an on-premise or cloud-hosted strategy.

Business Intelligence 276

Business Intelligence Analytics Testing Dashboards 276

CIO Business Intelligence

DECEMBER 17, 2024

What are the associated risks and costs, including operational, reputational, and competitive? Find a change champion and get business users involved from the beginning to build, pilot, test, and evaluate models. Consultants can help you develop and execute a genAI strategy that will fuel your success into 2025 and beyond.

Experimentation 98

Experimentation ROI Risk Data Quality 98

CIO Business Intelligence

OCTOBER 8, 2024

For those rare enterprises where innovation is more than a bullet point on a strategy statement embedded keep inside their SEC 10K, there is a repeatable approach for addressing the emerging unknown with great certainty. You risk adding to the hype where there will be no observable value. Test the customer waters.

Risk 124

Risk Technology Cost-Benefit Internet of Things 124

CIO Business Intelligence

JULY 30, 2024

Organizations are under pressure to demonstrate commitment to an actionable sustainability strategy to meet regulatory obligations and to build positive market sentiment. We examine the opportunity to lead both risk mitigation and value creation by helping advance the enterprise sustainability strategy.

Strategy 105

Strategy Scorecard Measurement Cost-Benefit 105

CIO Business Intelligence

JUNE 10, 2024

And they need people who can manage the emerging risks and compliance requirements associated with AI. Staffing strategies emerge Despite the continuously tight labor market and complexity of the task, Napoli believes he has Guardian Life’s AI talent strategy under control. Here’s how IT leaders are coping.

Strategy 136

Strategy IT Machine Learning Consulting 136

CIO Business Intelligence

MARCH 15, 2022

Synthetic data can also be a vital tool for enterprise AI efforts when available data doesn’t meet business needs or could create privacy issues if used to train machine learning models, test software, or the like. Artificial data has many uses in enterprise AI strategies. Synthetic data use cases. Synthetic data generation.

Strategy 145

Strategy Testing Sales Modeling 145

CIO Business Intelligence

JANUARY 30, 2025

CIOs are now reassessing the strategies to transform their organizations with gen AI, but its not exactly time to throw out the work thats already been done. That echoes a statement issued by NVIDIA on Monday: DeepSeek is a perfect example of test time scaling.

Modeling 117

Modeling Data-driven Technology Strategy 117

CIO Business Intelligence

JANUARY 30, 2025

particular, companies that use AI systems can share their voluntary commitments to transparency and risk control. At least half of the current AI Pact signatories (numbering more than 130) have made additional commitments, such as risk mitigation, human oversight and transparency in generative AI content.

Enterprise 128

Enterprise Risk Reporting Measurement 128