O'Reilly on Data

FEBRUARY 11, 2020

Almost half (48%) of respondents say they use data analysis, machine learning, or AI tools to address data quality issues. These include the basics, such as metadata creation and management, data provenance, data lineage, and other essentials. Just 20% of organizations publish data provenance and data lineage.

Data Quality 363

Data Quality Metadata Data Governance Publishing 363

Cloudera

MAY 19, 2021

In the previous blog post in this series, we walked through the steps for leveraging Deep Learning in your Cloudera Machine Learning (CML) projects. As a machine learning problem, it is a classification task with tabular data, a perfect fit for RAPIDS. Introduction. See < [link] > for more details.

Machine Learning 137

Machine Learning Data Science Data Lake Modeling 137

Rocket-Powered Data Science

JULY 6, 2021

This is accomplished through tags, annotations, and metadata (TAM). My favorite approach to TAM creation and to modern data management in general is AI and machine learning (ML). Smart content includes labeled (tagged, annotated) metadata (TAM). TAM management, like content management, begins with business strategy.

Strategy 267

Strategy Machine Learning Metadata Knowledge Discovery 267

CIO Business Intelligence

SEPTEMBER 6, 2024

As artificial intelligence (AI) and machine learning (ML) continue to reshape industries, robust data management has become essential for organizations of all sizes. Let’s dive into what that looks like, what workarounds some IT teams use today, and why metadata management is the key to success.

Metadata 116

Metadata Enterprise Management Cost-Benefit 116

Alation

JULY 19, 2021

What Is Metadata? Metadata is information about data. A clothing catalog or dictionary are both examples of metadata repositories. Indeed, a popular online catalog, like Amazon, offers rich metadata around products to guide shoppers: ratings, reviews, and product details are all examples of metadata.

Metadata 105

Metadata Management Data Governance Machine Learning 105

Analytics Vidhya

SEPTEMBER 17, 2022

Any type of contextual information, like device context, conversational context, and metadata, […]. However, we can improve the system’s accuracy by leveraging contextual information. The post Underlying Engineering Behind Alexa’s Contextual ASR appeared first on Analytics Vidhya.

Metadata 400

Metadata Statistics Data Science Publishing 400

O'Reilly on Data

MARCH 31, 2020

If you’re already a software product manager (PM), you have a head start on becoming a PM for artificial intelligence (AI) or machine learning (ML). AI products are automated systems that collect and learn from data to make user-facing decisions. We won’t go into the mathematics or engineering of modern machine learning here.

Management 363

Management Machine Learning Experimentation Metrics 363

O'Reilly on Data

JANUARY 29, 2019

In 2017, we published “ How Companies Are Putting AI to Work Through Deep Learning ,” a report based on a survey we ran aiming to help leaders better understand how organizations are applying AI through deep learning. We found companies were planning to use deep learning over the next 12-18 months.

Deep Learning 268

Deep Learning Machine Learning Data Science Metadata 268

AWS Big Data

JANUARY 9, 2025

Iceberg offers distinct advantages through its metadata layer over Parquet, such as improved data management, performance optimization, and integration with various query engines. Icebergs table format separates data files from metadata files, enabling efficient data modifications without full dataset rewrites.

Metadata 106

Metadata Snapshot Cost-Benefit Optimization 106

AWS Big Data

JANUARY 15, 2025

Improve accuracy and resiliency of analytics and machine learning by fostering data standards and high-quality data products. In addition to real-time analytics and visualization, the data needs to be shared for long-term data analytics and machine learning applications. This process is shown in the following figure.

IoT 100

IoT Machine Learning Metadata Data-driven 100

O'Reilly on Data

JANUARY 8, 2019

In a recent O’Reilly survey , we found that the skills gap remains one of the key challenges holding back the adoption of machine learning. For most companies, the road toward machine learning (ML) involves simpler analytic applications. Sustaining machine learning in an enterprise.

Machine Learning 271

Machine Learning IoT Internet of Things Data Science 271

AWS Big Data

JULY 29, 2024

This enables more informed decision-making and innovative insights through various analytics and machine learning applications. In this blog post, we’ll discuss how the metadata layer of Apache Iceberg can be used to make data lakes more efficient.

Metadata 118

Metadata Snapshot Data Lake Metrics 118

David Menninger's Analyst Perspectives

SEPTEMBER 28, 2021

Collibra is a data governance software company that offers tools for metadata management and data cataloging. The software enables organizations to find data quickly, identify its source and assure its integrity. Line-of-business workers can use it to create, review and update the organization's policies on different data assets.

Data Governance 349

Data Governance Metadata Software Management 349

Octopai

DECEMBER 12, 2021

Metadata is the pertinent, practical details about data assets: what they are, what to use them for, what to use them with. Without metadata, data is just a heap of numbers and letters collecting dust. Where does metadata come from? What is a metadata management tool? What are examples of metadata management tools?

Metadata 98

Metadata Management Data Quality Data Governance 98

AWS Big Data

NOVEMBER 26, 2024

Central to a transactional data lake are open table formats (OTFs) such as Apache Hudi , Apache Iceberg , and Delta Lake , which act as a metadata layer over columnar formats. In practice, OTFs are used in a broad range of analytical workloads, from business intelligence to machine learning.

Metadata 94

Metadata Data Lake Snapshot Data Warehouse 94

TDAN

JULY 20, 2021

Using machine learning and AI, Spotify creates value for their users by providing a more personalized experience. How does Spotify win against a competitor like Apple? They use data better.

Metadata 119

Metadata Machine Learning Data Architecture Data Strategy 119

Rocket-Powered Data Science

MARCH 10, 2020

The book Graph Algorithms: Practical Examples in Apache Spark and Neo4j is aimed at broadening our knowledge and capabilities around these types of graph analyses, including algorithms, concepts, and practical machine learning applications of the algorithms.

Metadata 250

Metadata Machine Learning Prescriptive Analytics ROI 250

CIO Business Intelligence

NOVEMBER 27, 2024

The analytics that drive AI and machine learning can quickly become compliance liabilities if security, governance, metadata management, and automation aren’t applied cohesively across every stage of the data lifecycle and across all environments.

Data Governance 132

Data Governance Risk Insurance Metadata 132

Octopai

OCTOBER 18, 2021

First, what active metadata management isn’t : “Okay, you metadata! Now, what active metadata management is (well, kind of): “Okay, you metadata! Metadata are the details on those tools: what they are, what to use them for, what to use them with. . That takes active metadata management. Quit lounging around!

Metadata 98

Metadata Management IT Data Quality 98

O'Reilly on Data

JUNE 19, 2019

A look at the landscape of tools for building and deploying robust, production-ready machine learning models. Our surveys over the past couple of years have shown growing interest in machine learning (ML) among organizations from diverse industries. Metadata and artifacts needed for a full audit trail.

Modeling 249

Modeling Machine Learning Testing Metrics 249

O'Reilly on Data

MARCH 20, 2019

Apply fair and private models, white-hat and forensic model debugging, and common sense to protect machine learning models from malicious actors. Like many others, I’ve known for some time that machine learning models themselves could pose security risks. Data poisoning attacks. General concerns.

Modeling 278

Modeling Machine Learning Predictive Modeling Consulting 278

AWS Big Data

NOVEMBER 7, 2024

Amazon Q generative SQL for Amazon Redshift uses generative AI to analyze user intent, query patterns, and schema metadata to identify common SQL query patterns directly within Amazon Redshift, accelerating the query authoring process for users and reducing the time required to derive actionable data insights.

Metadata 96

Metadata Sales Data Warehouse Optimization 96

IBM Big Data Hub

MAY 20, 2024

Machine learning (ML) has become a critical component of many organizations’ digital transformation strategy. In this blog post, we will explore the importance of lineage transparency for machine learning data sets and how it can help establish and ensure, trust and reliability in ML conclusions.

Machine Learning 87

Machine Learning Modeling Metadata Strategy 87

Cloudera

FEBRUARY 16, 2022

In this example, the Machine Learning (ML) model struggles to differentiate between a chihuahua and a muffin. We will learn what it is, why it is important and how Cloudera Machine Learning (CML) is helping organisations tackle this challenge as part of the broader objective of achieving Ethical AI.

Machine Learning 85

Machine Learning Modeling Metadata Recreation/Entertainment 85

Rocket-Powered Data Science

FEBRUARY 15, 2023

Most of these rules focus on the data, since data is ultimately the fuel, the input, the objective evidence, and the source of informative signals that are fed into all data science, analytics, machine learning, and AI models. FUD occurs when there is too much hype and “management speak” in the discussions.

Strategy 290

Strategy Experimentation Uncertainty Machine Learning 290

Smarten

NOVEMBER 26, 2024

The business can harness the power of statistics and machine learning to uncover those crucial nuggets of information that drive effective decision, and to improve the overall quality of data. Column Metadata – Provides information on the dataset’s recency, such as the last update and publication dates.

Machine Learning 64

Machine Learning Data Quality Predictive Modeling Metadata 64

AWS Big Data

OCTOBER 29, 2024

The CDH is used to create, discover, and consume data products through a central metadata catalog, while enforcing permission policies and tightly integrating data engineering, analytics, and machine learning services to streamline the user journey from data to insight.

Data Lake 88

Data Lake Sales Metadata Machine Learning 88

AWS Big Data

NOVEMBER 11, 2024

For example, you can use metadata about the Kinesis data stream name to index by data stream ( ${getMetadata("kinesis_stream_name") ), or you can use document fields to index data depending on the CloudWatch log group or other document data ( ${path/to/field/in/document} ).

Metadata 108

Metadata Metrics Analytics Data Processing 108

Cloudera

AUGUST 24, 2022

Cloudera Machine Learning (CML) is a cloud-native and hybrid-friendly machine learning platform. CML empowers organizations to build and deploy machine learning and AI capabilities for business at scale, efficiently and securely, anywhere they want. Cloudera Machine Learning. References.

Machine Learning 83

Machine Learning Metadata Testing Data Science 83

AWS Big Data

NOVEMBER 19, 2024

Solution overview By combining the powerful vector search capabilities of OpenSearch Service with the access control features provided by Amazon Cognito , this solution enables organizations to manage access controls based on custom user attributes and document metadata. If you don’t already have an AWS account, you can create one.

Management 93

Management Metadata Manufacturing Testing 93

CIO Business Intelligence

APRIL 8, 2025

As data-centric AI, automated metadata management and privacy-aware data sharing mature, the opportunity to embed data quality into the enterprises core has never been more significant. Data fabric Metadata-rich integration layer across distributed systems. Implementation complexity, relies on robust metadata management.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

AWS Big Data

DECEMBER 4, 2024

The Institutional Data & AI platform adopts a federated approach to data while centralizing the metadata to facilitate simpler discovery and sharing of data products. A data portal for consumers to discover data products and access associated metadata. Subscription workflows that simplify access management to the data products.

Metadata 93

Metadata Data Governance Data Quality Data-driven 93

AWS Big Data

NOVEMBER 20, 2024

We’re excited to announce a new feature in Amazon DataZone that offers enhanced metadata governance for your subscription approval process. With this update, domain owners can define and enforce metadata requirements for data consumers when they request access to data assets. Key benefits The feature benefits multiple stakeholders.

Metadata 75

Metadata Data Governance Metrics Marketing 75

CIO Business Intelligence

JANUARY 9, 2025

Before LLMs and diffusion models, organizations had to invest a significant amount of time, effort, and resources into developing custom machine-learning models to solve difficult problems. In many cases, this eliminates the need for specialized teams, extensive data labeling, and complex machine-learning pipelines.

Software 128

Software Enterprise Key Performance Indicator Machine Learning 128

CIO Business Intelligence

MAY 24, 2022

The industry-focused products look to solve the challenges of unstructured and siloed data by combining machine learning capabilities with specific integrations that the company calls “accelerators,” while complying with a variety of regulations and data standards. Intelligent Data Management Cloud for Health and Life Sciences.

Finance 140

Finance Management Metadata Machine Learning 140

CIO Business Intelligence

SEPTEMBER 19, 2024

This enables companies to directly access key metadata (tags, governance policies, and data quality indicators) from over 100 data sources in Data Cloud, it said. Additional to that, we are also allowing the metadata inside of Alation to be read into these agents.” That work takes a lot of machine learning and AI to accomplish.

Data Governance 123

Data Governance Metadata Unstructured Data Structured Data 123

AWS Big Data

FEBRUARY 6, 2023

AWS Glue is a serverless data integration service that makes it simple to discover, prepare, move, and integrate data from multiple sources for analytics, machine learning (ML), and application development. Choose the table to view the schema and other metadata.

Metadata 82

Metadata Data Lake Machine Learning Big Data 82