O'Reilly on Data

MARCH 25, 2025

Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. What breaks your app in production isnt always what you tested for in dev! The way out?

Testing 168

Testing Data-driven Software Measurement 168

CIO Business Intelligence

APRIL 8, 2025

Rule 1: Start with an acceptable risk appetite level Once a CIO understands their organizations risk appetite, everything else strategy, innovation, technology selection can align smoothly, says Paola Saibene, principal consultant at enterprise advisory firm Resultant. Most important, this plan should be tested and refined regularly.

Risk Management 131

Risk Management Risk Management Business Objectives 131

O'Reilly on Data

APRIL 25, 2023

First, note the overall strategy Xu Hao uses to write this code. He is using a strategy called “Knowledge Generation.” Many of the prompts are about testing: ChatGPT is instructed to generate tests for each function that it generates. If AI systems write the tests, do those tests themselves need to be tested?

Testing 364

Testing Software Strategy Enterprise 364

CIO Business Intelligence

MARCH 25, 2025

The proof of concept (POC) has become a key facet of CIOs AI strategies, providing a low-stakes way to test AI use cases without full commitment. The high number of Al POCs but low conversion to production indicates the low level of organizational readiness in terms of data, processes and IT infrastructure, IDCs authors report.

ROI 127

ROI IT Experimentation Risk 127

DataKitchen

APRIL 13, 2021

Testing and Data Observability. It orchestrates complex pipelines, toolchains, and tests across teams, locations, and data centers. Prefect Technologies — Open-source data engineering platform that builds, tests, and runs data workflows. Testing and Data Observability. Production Monitoring and Development Testing.

Testing 300

Testing Machine Learning Consulting Data Science 300

CIO Business Intelligence

JANUARY 7, 2025

But this year three changes are likely to drive CIOs operating model transformations and digital strategies: In 2024, enterprise SaaS embedded AI agents to drive workflow evolutions , and leading-edge organizations began developing their own AI agents.

Digital Transformation 143

Digital Transformation Experimentation Cost-Benefit Strategy 143

CIO Business Intelligence

NOVEMBER 14, 2023

Although some continue to leap without looking into cloud deals, the value of developing a comprehensive cloud strategy has become evident. Without a clear cloud strategy and broad leadership support, even value-adding cloud investments may be at risk. And it’s never too late for CIOs to reassess their cloud strategies.

Strategy 145

Strategy Cost-Benefit IT Key Performance Indicator 145

AWS Big Data

JANUARY 9, 2025

In our previous post Backtesting index rebalancing arbitrage with Amazon EMR and Apache Iceberg , we showed how to use Apache Iceberg in the context of strategy backtesting. This capability is particularly valuable in maintaining the integrity of backtests and the reliability of trading strategies.

Metadata 106

Metadata Snapshot Cost-Benefit Optimization 106

CIO Business Intelligence

JANUARY 14, 2025

By articulating fitness functions automated tests tied to specific quality attributes like reliability, security or performance teams can visualize and measure system qualities that align with business goals. Experimentation: The innovation zone Progressive cities designate innovation districts where new ideas can be tested safely.

Enterprise 119

Enterprise Technology Metrics Measurement 119

CIO Business Intelligence

JUNE 13, 2024

Not instant perfection The NIPRGPT experiment is an opportunity to conduct real-world testing, measuring generative AI’s computational efficiency, resource utilization, and security compliance to understand its practical applications. For now, AFRL is experimenting with self-hosted open-source LLMs in a controlled environment.

Testing 119

Testing Experimentation Data Processing Modeling 119

CIO Business Intelligence

JANUARY 13, 2025

There have long been data-driven CX strategies, but never with the autonomous power, or granular insights, that AI and new levels of predictive analytics will deliver in 2025. Advances in AI and ML will automate the compliance, testing, documentation and other tasks which can occupy 40-50% of a developers time.

Technology 133

Technology Interactive Cost-Benefit Testing 133

CIO Business Intelligence

APRIL 1, 2024

Here veteran IT leaders and advisers offer eight strategies to speed up IT modernization. CIOs can create dedicated testing units to test the output generated by gen AI LLMs, while establishing change management and upskilling processes to enable the workforce to maximize productivity throughout the modernization cycle.”

Strategy 140

Strategy IT Digital Transformation ROI 140

datapine

APRIL 15, 2020

When it comes to implementing and managing a successful BI strategy we have always proclaimed: start small, use the right BI tools , and involve your team. Your Chance: Want to test an agile business intelligence solution? You need to determine if you are going with an on-premise or cloud-hosted strategy.

Business Intelligence 276

Business Intelligence Analytics Testing Dashboards 276

CIO Business Intelligence

JUNE 10, 2024

Staffing strategies emerge Despite the continuously tight labor market and complexity of the task, Napoli believes he has Guardian Life’s AI talent strategy under control. He wants data scientists who can build, train, and validate models for use cases, and who can perform exploratory analysis and hypothesis testing.

Strategy 136

Strategy IT Machine Learning Consulting 136

CIO Business Intelligence

OCTOBER 23, 2024

In a survey of 451 senior technology executives conducted by Gartner in mid-2024, a striking 57% of CIOs reported being tasked with leading AI strategies. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

CIO Business Intelligence

DECEMBER 2, 2024

Suboptimal integration strategies are partly to blame, and on top of this, companies often don’t have security architecture that can handle both people and AI agents working on IT systems. If they’re going to benefit from AI strategies, companies must address this foundation before they can effectively scale their gen AI initiatives.

ROI 136

ROI Cost-Benefit Metrics Strategy 136

CIO Business Intelligence

NOVEMBER 4, 2024

Rather than wait for a storm to hit, IT professionals map out options and build strategies to ensure business continuity. This may involve embracing redundancies or testing new tools for future operations. The disruption from VMware’s acquisition has led many to reconsider their virtualization strategies and explore new options. “By

IT 124

IT Forecasting Strategy Testing 124

CIO Business Intelligence

JULY 23, 2024

So large corporations are still being put to the test with their implementing processes. There’s a big difference between sectors and organizations in terms of the level of maturity of their strategies. But it’s still early days since ChatGPT burst on the scene in 2022.

Strategy 115

Strategy Digital Transformation Technology Business Intelligence 115

CIO Business Intelligence

FEBRUARY 20, 2025

However, this perception of resilience must be backed up by robust, tested strategies that can withstand real-world threats. One major gap in the findings is that four in ten respondents admitted their organization had not reviewed its cyber resilience strategy in the last six months.

IT 98

IT Testing Strategy Data Processing 98

CIO Business Intelligence

APRIL 8, 2025

I aim to outline pragmatic strategies to elevate data quality into an enterprise-wide capability. This challenge remains deceptively overlooked despite its profound impact on strategy and execution. In this article, I am drawing from firsthand experience working with CIOs, CDOs, CTOs and transformation leaders across industries.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

CIO Business Intelligence

FEBRUARY 6, 2025

As someone deeply involved in shaping data strategy, governance and analytics for organizations, Im constantly working on everything from defining data vision to building high-performing data teams. with over 15 years of experience in enterprise data strategy, governance and digital transformation.

Unstructured Data 115

Unstructured Data Manufacturing Data Governance Sales 115

DataKitchen

JANUARY 6, 2021

An effective DataOps strategy can help a team invert this ratio and provide more value to the company. . Instead, these organizations commit 20% of their time implementing automation and writing tests. If it can be wrong, test it. Figure 1: Data professionals spend only 22% of their time on innovation. About the Author.

Cost-Benefit 246

Cost-Benefit Testing Strategy Metrics 246

CIO Business Intelligence

NOVEMBER 22, 2024

Unfortunately, despite hard-earned lessons around what works and what doesn’t, pressure-tested reference architectures for gen AI — what IT executives want most — remain few and far between, she said. “What’s Next for GenAI in Business” panel at last week’s Big.AI@MIT

Unstructured Data 136

Unstructured Data Testing Modeling Enterprise 136

CIO Business Intelligence

OCTOBER 24, 2024

Jayesh Chaurasia, analyst, and Sudha Maheshwari, VP and research director, wrote in a blog post that businesses were drawn to AI implementations via the allure of quick wins and immediate ROI, but that led many to overlook the need for a comprehensive, long-term business strategy and effective data management practices.

ROI 127

ROI Data-driven Enterprise Experimentation 127

CIO Business Intelligence

DECEMBER 18, 2024

Trading: GenAI optimizes quant finance, helps refine trading strategies, executes trades more effectively, and revolutionizes capital markets forecasting. Using deep neural networks and Azure GPUs built with NVIDIA technology, startup Riskfuel is developing accelerated models based on AI to determine derivative valuation and risk sensitivity.

Forecasting 105

Forecasting Predictive Analytics Risk Finance 105

O'Reilly on Data

DECEMBER 12, 2019

In addition to newer innovations, the practice borrows from model risk management, traditional model diagnostics, and software testing. Because ML models can react in very surprising ways to data they’ve never seen before, it’s safest to test all of your ML models with sensitivity analysis. [9] What can you do? Data augmentation.

Machine Learning 276

Machine Learning Modeling Testing Risk Management 276

CIO Business Intelligence

NOVEMBER 25, 2024

It’s an iterative process that involves regular monitoring, testing, and refining to make sure the AI is always working with the best possible data. From curation to integration, we help you align your data strategy with your AI goals. Looking to enhance the impact of your AI investments?

Data Quality 99

Data Quality ROI Interactive Modeling 99

DataKitchen

SEPTEMBER 17, 2021

Design your data analytics workflows with tests at every stage of processing so that errors are virtually zero in number. It’s hard enough to test within a single domain, but imagine testing with other domains which use different teams and toolchains, managed in other locations. Take a broader view.

Testing 237

Testing Data Architecture Measurement Visualization 237

CIO Business Intelligence

JANUARY 6, 2025

Deliver value from generative AI As organizations move from experimenting and testing generative AI use cases , theyre looking for gen AI to deliver real business value. Ronda Cilsick, CIO of software company Deltek, is aiming to do just that. As we go into 2025, well continue to see the evolution of gen AI.

IT 140

IT Cost-Benefit Measurement Experimentation 140

CIO Business Intelligence

AUGUST 28, 2024

For the technical architecture, we use a cloud-only strategy. Instead of painting 10 test panels, the body shop just needs two. For the back office, we’re consolidating several dozen ERP systems into a single instance supported by a global process template for the entire enterprise, starting with finance processes.

Strategy 121

Strategy B2B Finance Enterprise 121

datapine

AUGUST 12, 2020

Whether you manage a big or small company, business reports must be incorporated to establish goals, track operations, and strategy, to get an in-depth view of the overall company state. Your Chance: Want to test professional business reporting software? Your Chance: Want to test professional business reporting software?

Reporting 297

Reporting Dashboards Visualization Cost-Benefit 297

CIO Business Intelligence

DECEMBER 18, 2024

CIOs are facing these challenges head-on by designing integrated resilience strategies to future-proof their organizations. The pandemic has further underscored the importance of resilience, prompting CIOs to prioritize not only immediate risk management but also long-term resilience strategies, says Rajavel.

Risk 105

Risk Risk Management Cost-Benefit Measurement 105

datapine

APRIL 13, 2021

Your Chance: Want to test a powerful agency analytics software? Agencies benefit from interactive dashboard tools to prove the success of their strategies and campaigns to clients. Your Chance: Want to test a powerful agency analytics software? Your Chance: Want to test a powerful agency analytics software?

Dashboards 256

Dashboards Reporting Analytics Cost-Benefit 256

AWS Big Data

DECEMBER 5, 2024

To address this, we used the AWS performance testing framework for Apache Kafka to evaluate the theoretical performance limits. We conducted performance and capacity tests on the test MSK clusters that had the same cluster configurations as our development and production clusters.

Metrics 78

Metrics Dashboards Testing Optimization 78