CIO Business Intelligence

JANUARY 6, 2025

One of them is Katherine Wetmur, CIO for cyber, data, risk, and resilience at Morgan Stanley. Wetmur says Morgan Stanley has been using modern data science, AI, and machine learning for years to analyze data and activity, pinpoint risks, and initiate mitigation, noting that teams at the firm have earned patents in this space.

IT 140

IT Cost-Benefit Measurement Experimentation 140

CIO Business Intelligence

MARCH 25, 2025

CIOs perennially deal with technical debts risks, costs, and complexities. While the impacts of legacy systems can be quantified, technical debt is also often embedded in subtler ways across the IT ecosystem, making it hard to account for the full list of issues and risks.

Risk 140

Risk Cost-Benefit Data-driven Digital Transformation 140

O'Reilly on Data

JUNE 15, 2023

Should we risk loss of control of our civilization?” If we want prosocial outcomes, we need to design and report on the metrics that explicitly aim for those outcomes and measure the extent to which they have been achieved. And they are stress testing and “ red teaming ” them to uncover vulnerabilities.

Metrics 359

Metrics Reporting Measurement Finance 359

CIO Business Intelligence

OCTOBER 23, 2024

Despite AI’s potential to transform businesses, many senior technology leaders find themselves wrestling with unpredictable expenses, uneven productivity gains, and growing risks as AI adoption scales, Gartner said. CIOs should create proofs of concept that test how costs will scale, not just how the technology works.”

Enterprise 141

Enterprise Cost-Benefit Broadcasting Risk 141

CIO Business Intelligence

SEPTEMBER 17, 2024

CISOs can only know the performance and maturity of their security program by actively measuring it themselves; after all, to measure is to know. However, CISOs aren’t typically measuring their security program proactively or methodically to understand their current security program.

Measurement 111

Measurement Metrics Risk Dashboards 111

DataKitchen

SEPTEMBER 20, 2021

The best way to ensure error-free execution of data production is through automated testing and monitoring. The DataKitchen Platform enables data teams to integrate testing and observability into data pipeline orchestrations. Automated tests work 24×7 to ensure that the results of each processing stage are accurate and correct.

Testing 246

Testing Metrics Measurement Dashboards 246

CIO Business Intelligence

APRIL 23, 2024

As CIO, you’re in the risk business. Or rather, every part of your responsibilities entails risk, whether you’re paying attention to it or not. There are, for example, those in leadership roles who, while promoting the value of risk-taking, also insist on “holding people accountable.” You can’t lose.

Risk 116

Risk IT Insurance Risk Management 116

O'Reilly on Data

JUNE 19, 2019

A catalog or a database that lists models, including when they were tested, trained, and deployed. A catalog of validation data sets and the accuracy measurements of stored models. Model operations, testing, and monitoring. Other noteworthy items include: Tools for continuous integration and continuous testing of models.

Modeling 249

Modeling Machine Learning Testing Metrics 249

CIO Business Intelligence

FEBRUARY 7, 2024

Key AI companies have told the UK government to speed up its safety testing for their systems, raising questions about future government initiatives that too may hinge on technology providers opening up generative AI models to tests before new releases hit the public.

Testing 124

Testing Reporting Risk Publishing 124

CIO Business Intelligence

MARCH 5, 2025

If they decide a project could solve a big enough problem to merit certain risks, they then make sure they understand what type of data will be needed to address the solution. The next thing is to make sure they have an objective way of testing the outcome and measuring success. But we dont ignore the smaller players.

Testing 99

Testing Measurement Technology Experimentation 99

Smart Data Collective

MAY 7, 2023

million —and organizations are constantly at risk of cyber-attacks and malicious actors. In order to protect your business from these threats, it’s essential to understand what digital transformation entails and how you can safeguard your company from cyber risks. What is cyber risk?

Digital Transformation 129

Digital Transformation Risk Strategy Testing 129

CIO Business Intelligence

APRIL 8, 2025

Fragmented systems, inconsistent definitions, legacy infrastructure and manual workarounds introduce critical risks. The decisions you make, the strategies you implement and the growth of your organizations are all at risk if data quality is not addressed urgently. Manual entries also introduce significant risks.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

CIO Business Intelligence

DECEMBER 20, 2023

This has spurred interest around understanding and measuring developer productivity, says Keith Mann, senior director, analyst, at Gartner. Therefore, engineering leadership should measure software developer productivity, says Mann, but also understand how to do so effectively and be wary of pitfalls.

Measurement 124

Measurement Software IT Metrics 124

O'Reilly on Data

DECEMBER 12, 2019

1] This includes C-suite executives, front-line data scientists, and risk, legal, and compliance personnel. These recommendations are based on our experience, both as a data scientist and as a lawyer, focused on managing the risks of deploying ML. 6] Debugging may focus on a variety of failure modes (i.e., Sensitivity analysis.

Machine Learning 276

Machine Learning Modeling Testing Risk Management 276

CIO Business Intelligence

AUGUST 21, 2024

It wasn’t just a single measurement of particulates,” says Chris Mattmann, NASA JPL’s former chief technology and innovation officer. “It It was many measurements the agents collectively decided was either too many contaminants or not.” They also had extreme measurement sensitivity. Adding smarter AI also adds risk, of course.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

MAY 24, 2024

GRC certifications validate the skills, knowledge, and abilities IT professionals have to manage governance, risk, and compliance (GRC) in the enterprise. Enter the need for competent governance, risk and compliance (GRC) professionals. What are GRC certifications? Why are GRC certifications important?

Risk 119

Risk Risk Management Management Enterprise 119

O'Reilly on Data

JULY 15, 2019

In recent posts, we described requisite foundational technologies needed to sustain machine learning practices within organizations, and specialized tools for model development, model governance, and model operations/testing/monitoring. Note that the emphasis of SR 11-7 is on risk management.). Sources of model risk.

Machine Learning 51

Machine Learning Management Enterprise Risk Management 51

CIO Business Intelligence

OCTOBER 17, 2024

Allegations of fraud and security risks The indictment details that the fraudulent certification, combined with misleading claims about the facility’s capabilities, led the SEC to award Jain’s company the contract in 2012. The scheme allegedly put the SEC’s data security and operational integrity at risk.

Broadcasting 122

Broadcasting Risk Reporting Measurement 122

CIO Business Intelligence

JANUARY 30, 2025

These changes can expose businesses to risks and vulnerabilities such as security breaches, data privacy issues and harm to the companys reputation. It also includes managing the risks, quality and accountability of AI systems and their outcomes. Essentially to match their IT goals with their business goals. AI governance.

Testing 93

Testing Metrics Cost-Benefit Modeling 93

CIO Business Intelligence

JUNE 13, 2024

Not instant perfection The NIPRGPT experiment is an opportunity to conduct real-world testing, measuring generative AI’s computational efficiency, resource utilization, and security compliance to understand its practical applications. For now, AFRL is experimenting with self-hosted open-source LLMs in a controlled environment.

Testing 119

Testing Experimentation Data Processing Modeling 119

CIO Business Intelligence

JUNE 21, 2023

At the same time, the threat landscape continues to evolve and cyber risk is escalating for all organizations. As cyber risk continues to escalate, CIOs and CISOs need to be just as nimble and methodical as their adversaries. Because industry tests often lack standardized measurement criteria, the results can vary wildly.

Testing 98

Testing Measurement ROI Risk 98

CIO Business Intelligence

OCTOBER 17, 2024

While tech debt refers to shortcuts taken in implementation that need to be addressed later, digital addiction results in the accumulation of poorly vetted, misused, or unnecessary technologies that generate costs and risks. million machines worldwide, serves as a stark reminder of these risks. Assume unknown unknowns.

Risk 108

Risk Experimentation Testing Strategy 108

O'Reilly on Data

SEPTEMBER 15, 2020

Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., Technical competence: Competence measures a team’s ability to successfully deliver on initiatives and projects. Technical competence results in reduced risk and uncertainty.

Advertising 362

Advertising Cost-Benefit ROI Machine Learning 362

CIO Business Intelligence

FEBRUARY 21, 2025

Integration with Oracles systems proved more complex than expected, leading to prolonged testing and spiraling costs, the report stated. Change requests affecting critical aspects of the solution were accepted late in the implementation cycle, creating unnecessary complexity and risk.

Reporting 90

Reporting Risk Testing Risk Management 90

CIO Business Intelligence

NOVEMBER 13, 2024

Managers tend to incentivize activity metrics and measure inputs versus outputs,” she adds. And we’re at risk of being burned out.” If there are tools that are vetted, safe, and don’t pose security risks, and I can play around with them at my discretion, and if it helps me do my job better — great,” Woolley says.

IT 141

IT Consulting ROI Cost-Benefit 141

CIO Business Intelligence

OCTOBER 8, 2024

You risk adding to the hype where there will be no observable value. The learning phase Two key grounding musts: Non-mission critical workloads and (public) data Internal/private (closed) exposure This ensures no corporate information or systems will be exposed to any form of risk. Test the customer waters.

Risk 124

Risk Technology Cost-Benefit Internet of Things 124

DataKitchen

NOVEMBER 20, 2021

A DataOps Engineer can make test data available on demand. We have automated testing and a system for exception reporting, where tests identify issues that need to be addressed. It then autogenerates QC tests based on those rules. You can track, measure and create graphs and reporting in an automated way.

Testing 246

Testing Machine Learning Data Warehouse Analytics 246

AWS Big Data

DECEMBER 7, 2023

In this post, we outline planning a POC to measure media effectiveness in a paid advertising campaign. We chose to start this series with media measurement because “Results & Measurement” was the top ranked use case for data collaboration by customers in a recent survey the AWS Clean Rooms team conducted.

Measurement 105

Measurement Advertising Testing Data-driven 105

CIO Business Intelligence

JANUARY 30, 2025

The aim is to provide a framework that encourages early implementation of some of the measures in the act and to encourage organizations to make public the practices and processes they are implementing to achieve compliance even before the statutory deadline.In

Enterprise 128

Enterprise Risk Reporting Measurement 128

Rocket-Powered Data Science

FEBRUARY 15, 2023

3) How do we get started, when, who will be involved, and what are the targeted benefits, results, outcomes, and consequences (including risks)? Keep it agile, with short design, develop, test, release, and feedback cycles: keep it lean, and build on incremental changes. Test early and often. Test and refine the chatbot.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

CIO Business Intelligence

MAY 23, 2024

There is measurable progress, however, as data from the company’s connected products are collected in its own platform, where customers have access to information via a portal. “In Everything from simple translation services to more advanced solutions for creating product catalogues or risk analyses,” says Nilsson.

Digital Transformation 111

Digital Transformation Measurement Cost-Benefit Data Warehouse 111

DataKitchen

NOVEMBER 18, 2021

Write tests that catch data errors. The system creates on-demand development environments, performs automated impact reviews, tests/validates new analytics, deploys with a click, automates orchestrations, and monitors data pipelines 24×7 for errors and drift. Automate manual processes. Implement DataOps methods.

Testing 246

Testing Data Governance Measurement Software 246

Corinium

JUNE 6, 2019

Regulations and compliance requirements, especially around pricing, risk selection, etc., Fractal’s recommendation is to take an incremental, test and learn approach to analytics to fully demonstrate the program value before making larger capital investments. Build multiple MVPs to test conceptually and learn from early user feedback.

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

CIO Business Intelligence

MARCH 4, 2025

In fact, successful recovery from cyberattacks and other disasters hinges on an approach that integrates business impact assessments (BIA), business continuity planning (BCP), and disaster recovery planning (DRP) including rigorous testing. See also: How resilient CIOs future-proof to mitigate risks.)

Testing 64

Testing Risk Consulting Risk Management 64

Cloudera

JUNE 29, 2023

To ensure the stability of the US financial system, the implementation of advanced liquidity risk models and stress testing using (MI/AI) could potentially serve as a protective measure. To improve the way they model and manage risk, institutions must modernize their data management and data governance practices.

Data Architecture 85

Data Architecture Risk Management Risk Management 85

CIO Business Intelligence

SEPTEMBER 10, 2024

The proposed rules would require companies to report on development activities, cybersecurity measures, and results from red-teaming tests, which assess risks such as AI systems aiding cyberattacks or enabling non-experts to create chemical, biological, radiological, or nuclear weapons.

Reporting 131

Reporting Measurement Testing Risk 131

CIO Business Intelligence

FEBRUARY 20, 2025

If a database already exists, the available data must be tested and corrected. Solid reporting provides transparent, consistent and combined HR metrics essential for strategic planning, risk management and the management of HR measures. Companies should then monitor the measures and adjust them as necessary.

Big Data 84

Big Data Measurement Visualization Machine Learning 84