CIO Business Intelligence

OCTOBER 9, 2024

Depending on your needs, large language models (LLMs) may not be necessary for your operations, since they are trained on massive amounts of text and are largely for general use. As a result, they may not be the most cost-efficient AI model to adopt, as they can be extremely compute-intensive.

Strategy 143

Strategy Modeling Management Data Lake 143

O'Reilly on Data

OCTOBER 19, 2021

Let’s start by considering the job of a non-ML software engineer: writing traditional software deals with well-defined, narrowly-scoped inputs, which the engineer can exhaustively and cleanly model in the code. Not only is data larger, but models—deep learning models in particular—are much larger than before.

IT 350

IT Testing Experimentation Software 350

O'Reilly on Data

MAY 14, 2020

Experimentation: It’s just not possible to create a product by building, evaluating, and deploying a single model. In reality, many candidate models (frequently hundreds or even thousands) are created during the development process. Modelling: The model is often misconstrued as the most important component of an AI product.

Management 363

Management Experimentation B2B Machine Learning 363

datapine

OCTOBER 7, 2020

What gets measured gets done.” – Peter Drucker. By setting operational performance measures, you will know what is happening at every stage of your business. Since every business is different, it is essential to establish specific metrics and KPIs to measure, follow, calculate, and evaluate. Who will measure it?

KPI 269

KPI Metrics Sales Visualization 269

O'Reilly on Data

JUNE 15, 2023

The world changed on November 30, 2022 as surely as it did on August 12, 1908 when the first Model T left the Ford assembly line. If we want prosocial outcomes, we need to design and report on the metrics that explicitly aim for those outcomes and measure the extent to which they have been achieved.

Metrics 289

Metrics Reporting Measurement Finance 289

O'Reilly on Data

SEPTEMBER 15, 2020

In my book, I introduce the Technical Maturity Model: I define technical maturity as a combination of three factors at a given point of time. Technical sophistication: Sophistication measures a team’s ability to use advanced tools and techniques (e.g., PyTorch, TensorFlow, reinforcement learning, self-supervised learning).

Advertising 299

Advertising Cost-Benefit ROI Machine Learning 299

O'Reilly on Data

NOVEMBER 13, 2018

Considerations for a world where ML models are becoming mission critical. As the data community begins to deploy more machine learning (ML) models, I wanted to review some important considerations. Before I continue, it’s important to emphasize that machine learning is much more than building models. Model lifecycle management.

Machine Learning 223

Machine Learning Risk Management Statistics 223

DataKitchen

APRIL 13, 2021

DataOps needs a directed graph-based workflow that contains all the data access, integration, model and visualization steps in the data analytic production process. Observe, optimize, and scale enterprise data pipelines. . DataOps requires that teams measure their analytic processes in order to see how they are improving over time.

Testing 312

Testing Machine Learning Consulting Data Quality 312

Rocket-Powered Data Science

DECEMBER 16, 2018

In a related post we discussed the Cold Start Problem in Data Science — how do you start to build a model when you have either no training data or no clear choice of model parameters. We need to decide on all of these parameterizations of the clustering model before the cold start interations on the cluster means can begin.

Machine Learning 203

Machine Learning Metrics Optimization Data Science 203

O'Reilly on Data

MARCH 31, 2020

Instead of writing code with hard-coded algorithms and rules that always behave in a predictable manner, ML engineers collect a large number of examples of input and output pairs and use them as training data for their models. The model is produced by code, but it isn’t code; it’s an artifact of the code and the training data.

Management 338

Management Machine Learning Experimentation Metrics 338

KDnuggets

SEPTEMBER 20, 2022

When building and optimizing your classification model, measuring how accurately it predicts your expected outcome is crucial. That's where these additional performance evaluations come into play to help tease out more meaning from your model.

Metrics 160

Metrics Measurement Optimization Modeling 160

datapine

MAY 2, 2023

You can use big data analytics in logistics, for instance, to optimize routing, improve factory processes, and create razor-sharp efficiency across the entire supply chain. According to studies, 92% of data leaders say their businesses saw measurable value from their data and analytics investments.

Big Data 275

Big Data Internet of Things Cost-Benefit Optimization 275

CIO Business Intelligence

JULY 23, 2023

Optimization also rose to the top of IT leaders’ lists: 67% measure success within their IT organization by better optimizing resources. Optimizing IT resources—namely infrastructure, processes, and people—fuels digital transformation and modernization, which drives businesses to keep pace in today’s tough economy.

Optimization 95

Optimization IT Digital Transformation Uncertainty 95

DataKitchen

SEPTEMBER 20, 2021

The hours formerly wasted on unplanned work can be put to more productive use – creating innovative analytics for the enterprise and improving productivity further by investing in DataOps process optimizations. For example, data cleansing, ETL, running a model, or even provisioning cloud infrastructure. Measurement DataOps.

Testing 246

Testing Metrics Measurement Dashboards 246

IBM Big Data Hub

NOVEMBER 6, 2023

AI and API security Among existing API security measures, AI has emerged as a new — and potentially powerful — tool for fortifying APIs. AI technologies can also enable automated threat modeling. Using historical API data, AI can build threat models to predict vulnerabilities and threats before bad actors can exploit them.

Optimization 113

Optimization Testing Software Risk 113

O'Reilly on Data

JUNE 18, 2019

There has been a significant increase in our ability to build complex AI models for predictions, classifications, and various analytics tasks, and there’s an abundance of (fairly easy-to-use) tools that allow data scientists and analysts to provision complex models within days. Data integration and cleaning.

Machine Learning 263

Machine Learning Data Quality Statistics Modeling 263

datapine

DECEMBER 11, 2019

SaaS is a software distribution model that offers a lot of agility and cost-effectiveness for companies, which is why it’s such a reliable option for numerous business models and industries. AI optimizes business processes, increasing productivity and efficiency while automating repetitive tasks and supporting human capabilities.

Software 314

Software Cost-Benefit Data-driven Data Processing 314

O'Reilly on Data

JUNE 18, 2024

Last month, TheNew York Times claimed that tech giants OpenAI and Google have waded into a copyright gray area by transcribing the vast volume of YouTube videos and using that text as additional training data for their AI models despite terms of service that prohibit such efforts and copyright law that the Times argues places them in dispute.

Advertising 353

Advertising Modeling Publishing Data Processing 353

IBM Big Data Hub

MAY 17, 2024

Cloud maturity models are a useful tool for addressing these concerns, grounding organizational cloud strategy and proceeding confidently in cloud adoption with a plan. Cloud maturity models (or CMMs) are frameworks for evaluating an organization’s cloud adoption readiness on both a macro and individual service level.

Modeling 96

Modeling Cost-Benefit Optimization Digital Transformation 96

O'Reilly on Data

AUGUST 4, 2022

We can find many more examples across many more decades that reflect naiveté and optimism and–if we are honest–no small amount of ignorance and hubris. This kind of humility is likely to deliver more meaningful progress and a more measured understanding of such progress. We typically underappreciate how complex such systems are.

Machine Learning 268

Machine Learning Modeling Statistics Software 268

O'Reilly on Data

JULY 15, 2019

In recent posts, we described requisite foundational technologies needed to sustain machine learning practices within organizations, and specialized tools for model development, model governance, and model operations/testing/monitoring. Sources of model risk. Model risk management. Image by Ben Lorica.

Machine Learning 38

Machine Learning Management Enterprise Risk Management 38

DataKitchen

NOVEMBER 27, 2024

For example, metrics like the percentage of missing values help measure completeness, while deviations from authoritative sources gauge accuracy. For instance, a bank might use a CDE dashboard to monitor the accuracy of customer credit scores, which are vital for regulatory compliance and effective risk modeling.

Data Quality 219

Data Quality Dashboards Metrics Cost-Benefit 219

CIO Business Intelligence

OCTOBER 24, 2024

Take for instance large language models (LLMs) for GenAI. Businesses will need to invest in hardware and infrastructure that are optimized for AI and this may incur significant costs. Contextualizing patterns and identifying potential threats can minimize alert fatigue and optimize the use of resources.

Risk 128

Risk Measurement Optimization Machine Learning 128

CIO Business Intelligence

AUGUST 9, 2024

Twenty-nine percent of 644 executives at companies in the US, Germany, and the UK said they were already using gen AI, and it was more widespread than other AI-related technologies, such as optimization algorithms, rule-based systems, natural language processing, and other types of ML. A balance between privacy and utility is needed.

Risk 138

Risk Measurement Modeling Data Lake 138

O'Reilly on Data

MARCH 23, 2023

Many of these go slightly (but not very far) beyond your initial expectations: you can ask it to generate a list of terms for search engine optimization, you can ask it to generate a reading list on topics that you’re interested in. It’s important to understand that ChatGPT is not actually a language model. with specialized training.

IT 267

IT Modeling Testing Risk 267

datapine

JANUARY 24, 2021

Additionally, incorporating a decision support system software can save a lot of company’s time – combining information from raw data, documents, personal knowledge, and business models will provide a solid foundation for solving business problems. There are basically 4 types of scales: *Statistics Level Measurement Table*.

IT 317

IT Statistics KPI Data-driven 317

AWS Big Data

NOVEMBER 27, 2024

For instance, it’s used in generative AI applications to match user queries to semantically similar knowledge articles providing context and grounding for generative models to perform tasks. Therefore, cost optimization levers are important to achieve a favorable balance of cost vs. benefit.

Cost-Benefit 122

Cost-Benefit Optimization Machine Learning Software 122

CIO Business Intelligence

NOVEMBER 7, 2024

As digital transformation becomes a critical driver of business success, many organizations still measure CIO performance based on traditional IT values rather than transformative outcomes. This creates a disconnect between the strategic role that CIOs are increasingly expected to play and how their success is measured.

Digital Transformation 128

Digital Transformation Metrics Measurement Key Performance Indicator 128

CIO Business Intelligence

SEPTEMBER 26, 2024

Many developers say AI coding assistants make them more productive, but a recent study set forth to measure their output and found no significant gains. The study measured pull request (PR) cycle time, or the time to merge code into a repository, and PR throughput, the number of pull requests merged.

Measurement 145

Measurement Software Testing Data-driven 145

CIO Business Intelligence

JANUARY 10, 2024

Optimizing cloud investments requires close collaboration with the rest of the business to understand current and future needs, building effective FinOps teams, partnering with providers, and ongoing monitoring of key performance metrics. You worry you don’t have enough capacity, so you overprovision,” he says.

Cost-Benefit 124

Cost-Benefit Measurement Optimization Metrics 124

CIO Business Intelligence

NOVEMBER 19, 2024

Generative AI (GenAI) is rapidly emerging as a game changer for enterprises, but turning its potential into measurable value remains a significant challenge. Data privacy and compliance issues Failing: Mismanagement of internal data with external models can lead to privacy breaches and non-compliance with regulations.

Consulting 137

Consulting Cost-Benefit Experimentation Optimization 137

datapine

SEPTEMBER 29, 2022

5) How Do You Measure Data Quality? In this article, we will detail everything which is at stake when we talk about DQM: why it is essential, how to measure data quality, the pillars of good quality management, and some data quality control techniques. These needs are then quantified into data models for acquisition and delivery.

Data Quality 241

Data Quality Metrics Data-driven Management 241

datapine

FEBRUARY 4, 2020

Management thinker Peter Drucker once stated, “if you can’t measure it, you can’t improve it” – and he couldn’t be more right. If it has been optimized for SEO though, you shouldn’t stop measuring it after the first week, as it needs a couple of months to reach its “cruising traffic”, and you can get several thousands of monthly visits.

Reporting 280

Reporting Marketing Advertising Metrics 280

O'Reilly on Data

OCTOBER 11, 2022

Data integration problems aren’t pretty; they’re boring, uninteresting, the “killing field of any modeling project,” as Lorien Pratt has said. How can you use it to analyze your current situation, and measure the results of any actions you take? If you take some action, what changes? Most actions have multiple effects.

Data Integration 297

Data Integration Insurance Dashboards Modeling 297

CIO Business Intelligence

JULY 24, 2024

Large language models (LLMs) are hard to beat when it comes to instantly parsing reams of publicly available data to generate responses to general knowledge queries. Moreover, challenges around data privacy and recognition of intellectual property often require a level of transparency that simply does not exist in many off-the-shelf models.

Modeling 75

Modeling Insurance Cost-Benefit Unstructured Data 75

CIO Business Intelligence

FEBRUARY 26, 2024

Instead, many digitized analog-first processes and bolted them onto aging operating models. Companies will miss out if they make assumptions about business and operational models without reflecting on the possibilities afforded by AI. In short, modern operating models will be the enabler of business transformation.

Cost-Benefit 137

Cost-Benefit Modeling Data-driven Enterprise 137

Occam's Razor

JULY 5, 2021

Organic Search, got the individual to an optimal page on your site. Here’s how you can measure how sophisticated your attribution approach is: If you are using the full power of the attribution modeling across owned, earned, and paid, you are at an industry-average level of analytics sophistication.?. Absolutely!

Marketing Analytics 148

Marketing Analytics Marketing Analytics Data-driven 148