O'Reilly on Data

MARCH 23, 2023

It’s important to understand that ChatGPT is not actually a language model. It’s a convenient user interface built around one specific language model, GPT-3.5, is one of a class of language models that are sometimes called “large language models” (LLMs)—though that term isn’t very helpful. with specialized training.

IT 348

IT Modeling Testing Risk 348

Ontotext

JANUARY 10, 2025

Generative AI (GenAI) models, such as GPT-4, offer a promising solution, potentially reducing the dependency on labor-intensive annotation. Through iterative experimentation, we incrementally added new modules refining the prompts. BioRED performance Prompt Model P R F1 Price Latency Generic prompt GPT-4o 72 35 47.8

Informatics 59

Informatics Modeling Metadata Experimentation 59

AWS Big Data

DECEMBER 12, 2024

In the context of comprehensive data governance, Amazon DataZone offers organization-wide data lineage visualization using Amazon Web Services (AWS) services, while dbt provides project-level lineage through model analysis and supports cross-project integration between data lakes and warehouses.

Snapshot 79

Snapshot Recreation/Entertainment Experimentation Data Lake 79

CIO Business Intelligence

JUNE 21, 2024

And it enables research teams to analyze legislation and policy documents in record time, delivering plans for proposed changes to these critical agencies in a day rather than weeks. By June 2024, MITREChatGPT offered document analysis and reasoning on thousands of documents, provided an enterprise prompt library, and made GPT 3.5

Experimentation 134

Experimentation Data mining Reporting Data Processing 134

CIO Business Intelligence

MARCH 19, 2025

I first described the overall AI landscape and made sure they realized weve been doing AI for quite a while in the form of machine learning and other deterministic models. This enforces the need for good data governance, as AI models will surface incorrect data more frequently, and most likely at a greater cost to the business.

Metrics 64

Metrics Enterprise Cost-Benefit Experimentation 64

CIO Business Intelligence

AUGUST 14, 2024

It’s embedded in the applications we use every day and the security model overall is pretty airtight. Microsoft has also made investments beyond OpenAI, for example in Mistral and Meta’s LLAMA models, in its own small language models like Phi, and by partnering with providers like Cohere, Hugging Face, and Nvidia. That’s risky.”

Enterprise 136

Enterprise Cost-Benefit Experimentation Modeling 136

CIO Business Intelligence

SEPTEMBER 18, 2024

Even as it designs 3D generative AI models for future customer deployment, CAD/CAM design giant Autodesk is “leaning” into generative AI for its customer service operations, deploying Salesforce’s Einstein for Service with plans to use Agentforce in the future, CIO Prakash Kota says. Not anymore. “If

Digital Transformation 119

Digital Transformation Experimentation Sales Software 119

CIO Business Intelligence

AUGUST 20, 2024

Our mental models of what constitutes a high-performance team have evolved considerably over the past five years. Post-pandemic, high-performance teams excelled at remote and hybrid working models, were more empathetic to individual needs, and leveraged automation to reduce manual work.

IT 141

IT Digital Transformation Experimentation Risk 141

CIO Business Intelligence

OCTOBER 5, 2023

Sandeep Davé knows the value of experimentation as well as anyone. Davé and his team’s achievements in AI are due in large part to creating opportunities for experimentation — and ensuring those experiments align with CBRE’s business strategy. Let’s start with the models. And those experiments have paid off.

Experimentation 130

Experimentation Strategy Machine Learning Interactive 130

Rocket-Powered Data Science

OCTOBER 16, 2020

We build models to test our understanding, but these models are not “one and done.” In ML, the learning cycle is sometimes called backpropagation, where the errors (inaccurate predictions) of our models are fed back into adjusting the model’s input parameters in a way that aims to improve the output accuracy. That’s data.

Digital Transformation 142

Digital Transformation Experimentation Data Science Data Strategy 142

CIO Business Intelligence

MARCH 12, 2024

Early use cases include code generation and documentation, test case generation and test automation, as well as code optimization and refactoring, among others. Junior developers are reporting the biggest productivity boosts, but this remains an area of active research and experimentation,” Tandon says.

Software 131

Software Experimentation Testing Cost-Benefit 131

CIO Business Intelligence

MAY 9, 2024

Experimentation drives momentum: How do we maximize the value of a given technology? Via experimentation. This can be as simple as a Google Sheet or sharing examples at weekly all-hands meetings Many enterprises do “blameless postmortems” to encourage experimentation without fear of making mistakes and reprisal.

Enterprise 122

Enterprise Experimentation Sales Manufacturing 122

CIO Business Intelligence

MAY 1, 2024

Gen AI takes us from single-use models of machine learning (ML) to AI tools that promise to be a platform with uses in many areas, but you still need to validate they’re appropriate for the problems you want solved, and that your users know how to use gen AI effectively. Pilots can offer value beyond just experimentation, of course.

Cost-Benefit 143

Cost-Benefit Metrics Insurance Measurement 143

AWS Big Data

SEPTEMBER 5, 2024

In the context of Retrieval-Augmented Generation (RAG), knowledge retrieval plays a crucial role, because the effectiveness of retrieval directly impacts the maximum potential of large language model (LLM) generation. It comes in two modes: document-only and bi-encoder. You can get its model ID from the response.

Metrics 106

Metrics Testing Experimentation Modeling 106

Domino Data Lab

AUGUST 22, 2019

The excerpt covers how to create word vectors and utilize them as an input into a deep learning model. While the field of computational linguistics, or Natural Language Processing (NLP), has been around for decades, the increased interest in and use of deep learning models has also propelled applications of NLP forward within industry.

Deep Learning 111

Deep Learning Modeling Metrics Testing 111

CIO Business Intelligence

JULY 1, 2024

Notable examples of AI safety incidents include: Trading algorithms causing market “flash crashes” ; Facial recognition systems leading to wrongful arrests ; Autonomous vehicle accidents ; AI models providing harmful or misleading information through social media channels.

Reporting 129

Reporting Risk Management Experimentation Risk 129

CIO Business Intelligence

AUGUST 23, 2024

Data scientists at Bayer have developed several proofs of concept of generative AI models on the new platform that remain in discovery and evaluation phase for “efficacy,” McQueen says, adding that the models won’t be in production until 2025. The R&D pipeline is pretty highly confidential at this point,” he says.

Data Science 101

Data Science Experimentation Testing Modeling 101

DataRobot Blog

JANUARY 10, 2023

Most, if not all, machine learning (ML) models in production today were born in notebooks before they were put into production. Data science teams of all sizes need a productive, collaborative method for rapid AI experimentation. Capabilities Beyond Classic Jupyter for End-to-end Experimentation. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

CIO Business Intelligence

AUGUST 15, 2024

A virtual assistant may save employees time when searching for old documents or composing emails, but most organizations have no idea how much time those tasks have taken historically, having never tracked such metrics before, she says. However, foundational models will always have a place as the core backbone for the industry.”

ROI 143

ROI Cost-Benefit Experimentation Deep Learning 143

Dataiku

MARCH 25, 2024

This scenario is not science fiction but a glimpse into the capabilities of Multimodal Large Language Models (M-LLMs), where the convergence of various modalities extends the landscape of AI. But instead, a machine seamlessly identifies the scene and its location, provides a detailed description, and even suggests nearby attractions.

Visualization 132

Visualization Modeling Experimentation Testing 132

CIO Business Intelligence

NOVEMBER 21, 2023

Set parameters and emphasize collaboration To address one root cause of shadow IT, CIOs must also establish a governance and delivery model for evaluating, procuring, and implementing department technology solutions.

IT 137

IT Risk Cost-Benefit Data Science 137

Decision Management Solutions

MAY 12, 2021

by Charlotte DeKeyrel, Expert Decision Modeler. When decisions are properly documented and developed with rigor, everyone gets smarter by understanding the complexities and flow of decision-making. Decision Management Solutions. You know who they are—the members of the brain trust in your organizations.

Experimentation 52

Experimentation Insurance Modeling Management 52

CIO Business Intelligence

MARCH 7, 2023

Vince Kellen understands the well-documented limitations of ChatGPT, DALL-E and other generative AI technologies — that answers may not be truthful, generated images may lack compositional integrity, and outputs may be biased — but he’s moving ahead anyway. And, he says, using generative AI for coding has worked well.

Enterprise 116

Enterprise IT Unstructured Data Experimentation 116

DataRobot Blog

MARCH 16, 2023

Why model-driven AI falls short of delivering value Teams that just focus model performance using model-centric and data-centric ML risk missing the big picture business context. We are also thrilled to share the innovations and capabilities that we have developed at DataRobot to meet and exceed those requirements.

Experimentation 98

Experimentation Data-driven Modeling Metadata 98

CIO Business Intelligence

DECEMBER 19, 2023

Many technology investments are merely transitionary, taking something done today and upgrading it to a better capability without necessarily transforming the business or operating model. If CIOs don’t improve conversions from pilot to production, they may find their investors losing patience in the process and culture of experimentation.

Digital Transformation 143

Digital Transformation Unstructured Data Machine Learning Risk Management 143

Domino Data Lab

APRIL 7, 2019

Other use cases may involve returning the most appropriate answer to a question, finding the most relevant documents for a query or classifying the input document itself. A good NLP library will make it easy to both train your own NLP models and integrate with the downstream ML or DL pipeline. Training domain-specific models.

Deep Learning 110

Deep Learning Machine Learning Experimentation Modeling 110

AWS Big Data

JANUARY 9, 2024

Lexical search In lexical search, the search engine compares the words in the search query to the words in the documents, matching word for word. Traditional lexical search, based on term frequency models like BM25, is widely used and effective for many search applications. Only items that have words the user typed match the query.

Visualization 107

Visualization Cost-Benefit Modeling Machine Learning 107

Domino Data Lab

FEBRUARY 12, 2020

for model serving (experimental), are implemented with Ray internally for its scalable, distributed computing and state management benefits, while providing a domain-specific API for the purposes they serve. Motivations for Ray: Training a Reinforcement Learning (RL) Model. Handling Stateful Computation with an Actor Model.

Experimentation 137

Experimentation Modeling Data Science Machine Learning 137

AWS Big Data

AUGUST 21, 2023

Lexical search looks for words in the documents that appear in the queries. For the demo, we’re using the Amazon Titan foundation model hosted on Amazon Bedrock for embeddings, with no fine tuning. In lexical search, the search engine compares the words in the search query to the words in the documents, matching word for word.

Data Processing 96

Data Processing Visualization Experimentation Metrics 96

Domino Data Lab

AUGUST 6, 2021

Model Risk Management is about reducing bad consequences of decisions caused by trusting incorrect or misused model outputs. Systematically enabling model development and production deployment at scale entails use of an Enterprise MLOps platform, which addresses the full lifecycle including Model Risk Management.

Risk Management 64

Risk Management Risk Modeling Enterprise 64

CIO Business Intelligence

APRIL 14, 2023

In fact, it’s likely your organization has a large number of employees currently experimenting with generative AI, and as this activity moves from experimentation to real-life deployment, it’s important to be proactive before unintended consequences happen. For example, will this cover all forms of AI or just generative AI?

Risk 122

Risk Experimentation Marketing Risk Management 122

CIO Business Intelligence

JULY 3, 2023

But the rise of large language models (LLMs) is starting to make true knowledge management (KM) a reality. These models can extract meaning from digital data at scale and speed beyond the capabilities of human analysts. Data exists in ever larger silos, but real knowledge still resides in employees.

Management 105

Management Experimentation Modeling Enterprise 105

CIO Business Intelligence

APRIL 5, 2023

Midjourney, ChatGPT, Bing AI Chat, and other AI tools that make generative AI accessible have unleashed a flood of ideas, experimentation and creativity. That turns generic documentation into conversational programming where the AI can take your data and show you how to write a query, for example.

Modeling 111

Modeling Testing Cost-Benefit Experimentation 111

CIO Business Intelligence

APRIL 24, 2024

But to find ways it can help grow a company’s bottom line, CIOs have to do more to understand a company’s business model and identify opportunities where gen AI can change the playing field. We have a HITRUST certified health care environment and we bring in publicly-available models.” And there are audit trails for everything.”

Sales 128

Sales Finance Experimentation Cost-Benefit 128

CIO Business Intelligence

AUGUST 21, 2024

According to Gartner, an agent doesn’t have to be an AI model. Starting in 2018, the agency used agents, in the form of Raspberry PI computers running biologically-inspired neural networks and time series models, as the foundation of a cooperative network of sensors. “It And, yes, enterprises are already deploying them.

Risk 136

Risk Insurance Cost-Benefit Software 136

CIO Business Intelligence

FEBRUARY 7, 2024

We’ve seen an ongoing iteration of experimentation with a number of promising pilots in production,” he says. Samsara employees are applying these general-purpose assistants to a variety of use cases, like writing documentation and job descriptions, debugging code, or writing API endpoints.

KPI 105

KPI Measurement Experimentation Finance 105

CIO Business Intelligence

APRIL 2, 2024

Joanne Friedman, PhD, CEO, and principal of smart manufacturing at Connektedminds, says orchestrating success in digital transformation requires a symphony of integration across disciplines : “CIOs face the challenge of harmonizing diverse disciplines like design thinking, product management, agile methodologies, and data science experimentation.

Management 140

Management Digital Transformation Testing Experimentation 140