David Menninger's Analyst Perspectives

SEPTEMBER 28, 2021

Collibra is a data governance software company that offers tools for metadata management and data cataloging. The software enables organizations to find data quickly, identify its source and assure its integrity. Line-of-business workers can use it to create, review and update the organization's policies on different data assets.

Data Governance 349

Data Governance Metadata Software Management 349

CIO Business Intelligence

SEPTEMBER 6, 2024

Central to this is metadata management, a critical component for driving future success AI and ML need large amounts of accurate data for companies to get the most out of the technology. Let’s dive into what that looks like, what workarounds some IT teams use today, and why metadata management is the key to success.

Metadata 116

Metadata Enterprise Management Cost-Benefit 116

O'Reilly on Data

FEBRUARY 11, 2020

These include the basics, such as metadata creation and management, data provenance, data lineage, and other essentials. They’re still struggling with the basics: tagging and labeling data, creating (and managing) metadata, managing unstructured data, etc. Data quality might get worse before it gets better. Respondent demographics.

Data Quality 363

Data Quality Metadata Data Governance Publishing 363

AWS Big Data

JANUARY 9, 2025

Iceberg offers distinct advantages through its metadata layer over Parquet, such as improved data management, performance optimization, and integration with various query engines. Icebergs table format separates data files from metadata files, enabling efficient data modifications without full dataset rewrites.

Metadata 107

Metadata Snapshot Cost-Benefit Optimization 107

AWS Big Data

NOVEMBER 14, 2024

Under the hood, UniForm generates Iceberg metadata files (including metadata and manifest files) that are required for Iceberg clients to access the underlying data files in Delta Lake tables. Both Delta Lake and Iceberg metadata files reference the same data files.

Metadata 111

Metadata Data Warehouse Big Data Data Lake 111

AWS Big Data

JULY 29, 2024

In this blog post, we’ll discuss how the metadata layer of Apache Iceberg can be used to make data lakes more efficient. You will learn about an open-source solution that can collect important metrics from the Iceberg metadata layer. Data lakes provide a unified repository for organizations to store and use large volumes of data.

Metadata 119

Metadata Snapshot Data Lake Metrics 119

CIO Business Intelligence

DECEMBER 18, 2024

This ability builds on the deep metadata context that Salesforce has across a variety of tasks. Christened Agentforce 2.0, And the ability to build agents using natural language will extend the low code suites usability to a variety of users within an enterprise, Jyoti added. New agent skills in Agentforce 2.0

Enterprise 98

Enterprise IT Sales Metadata 98

AWS Big Data

NOVEMBER 26, 2024

Central to a transactional data lake are open table formats (OTFs) such as Apache Hudi , Apache Iceberg , and Delta Lake , which act as a metadata layer over columnar formats. XTable isn’t a new table format but provides abstractions and tools to translate the metadata associated with existing formats.

Metadata 95

Metadata Data Lake Snapshot Data Warehouse 95

DataKitchen

FEBRUARY 20, 2025

Better Metadata Management Add Descriptions and Data Product tags to tables and columns in the Data Catalog for improved governance. Imagine an open-source tool thats free to download but requires minimal time and effort. DataOps just got more intelligent.

Data Quality 183

Data Quality Scorecard Testing Dashboards 183

CIO Business Intelligence

FEBRUARY 27, 2025

According to a study from Rocket Software and Foundry , 76% of IT decision-makers say challenges around accessing mainframe data and contextual metadata are a barrier to mainframe data usage, while 64% view integrating mainframe data with cloud data sources as the primary challenge.

Metadata 98

Metadata Data Lake Cost-Benefit Forecasting 98

AWS Big Data

NOVEMBER 22, 2024

How RFS works OpenSearch and Elasticsearch snapshots are a directory tree that contains both data and metadata. Metadata files exist in the snapshot to provide details about the snapshot as a whole, the source cluster’s global metadata and settings, each index in the snapshot, and each shard in the snapshot.

Snapshot 82

Snapshot Metadata Recreation/Entertainment Data Processing 82

AWS Big Data

NOVEMBER 7, 2024

Amazon Q generative SQL for Amazon Redshift uses generative AI to analyze user intent, query patterns, and schema metadata to identify common SQL query patterns directly within Amazon Redshift, accelerating the query authoring process for users and reducing the time required to derive actionable data insights. This generates a SQL query.

Metadata 97

Metadata Sales Data Warehouse Optimization 97

CIO Business Intelligence

DECEMBER 11, 2024

One field that is gaining attention is data intelligence, which uses metadata to provide visibility and a deeper and broader understanding of data quality, context, usage, and impact. In fact, a data framework is critical first step for AI success. Yet research shows Australians are already using AI without formal policies.

Risk 111

Risk Data Strategy Strategy Data Governance 111

AWS Big Data

NOVEMBER 29, 2023

The Eightfold Talent Intelligence Platform integrates with Amazon Redshift metadata security to implement visibility of data catalog listing of names of databases, schemas, tables, views, stored procedures, and functions in Amazon Redshift. This post discusses restricting listing of data catalog metadata as per the granted permissions.

Metadata 111

Metadata Data Warehouse Analytics Data Analytics 111

CIO Business Intelligence

APRIL 8, 2025

As data-centric AI, automated metadata management and privacy-aware data sharing mature, the opportunity to embed data quality into the enterprises core has never been more significant. These issues dont just hinder next-gen analytics and AI; they erode trust, delay transformation and diminish business value.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

AWS Big Data

APRIL 9, 2025

Whether youre a data analyst seeking a specific metric or a data steward validating metadata compliance, this update delivers a more precise, governed, and intuitive search experience. Refer to the product documentation to learn more about how to set up metadata rules for subscription and publishing workflows.

Metadata 85

Metadata Metrics Cost-Benefit Data-driven 85

AWS Big Data

DECEMBER 4, 2024

In today’s rapidly evolving financial landscape, data is the bedrock of innovation, enhancing customer and employee experiences and securing a competitive edge. Like many large financial institutions, ANZ Institutional Division operated with siloed data practices and centralized data management teams.

Metadata 94

Metadata Data Governance Data Quality Data-driven 94

O'Reilly on Data

MARCH 22, 2019

We’re all used to spam blocking, and we don’t object to it, at least partly because email would be unusable without it. And blocking spam requires making ethical decisions automatically: deciding that a message is spam means deciding what other people can and can’t say, and who they can say it to. There’s a lot we can learn from spam filtering.

Metadata 275

Metadata Advertising Insurance Modeling 275

Cloudera

AUGUST 6, 2024

And for that future to be a reality, data teams must shift their attention to metadata, the new turf war for data. And for that future to be a reality, data teams must shift their attention to metadata, the new turf war for data. Data teams actually need to unify the metadata. Data teams manage this metadata with a metastore.

Metadata 82

Metadata Cost-Benefit Management Enterprise 82

CIO Business Intelligence

NOVEMBER 27, 2024

The analytics that drive AI and machine learning can quickly become compliance liabilities if security, governance, metadata management, and automation aren’t applied cohesively across every stage of the data lifecycle and across all environments. How does a business stand out in a competitive market with AI?

Data Governance 132

Data Governance Risk Insurance Metadata 132

AWS Big Data

JANUARY 15, 2025

From here, the metadata is published to Amazon DataZone by using AWS Glue Data Catalog. EUROGATE is a leading independent container terminal operator in Europe, known for its reliable and professional container handling services. Their terminal operations rely heavily on seamless data flows and the management of vast volumes of data.

IoT 101

IoT Machine Learning Metadata Data-driven 101

AWS Big Data

DECEMBER 4, 2024

An Iceberg table’s metadata stores a history of snapshots, which are updated with each transaction. Over time, this creates multiple data files and metadata files as changes accumulate. Additionally, they can impact query performance due to the overhead of handling large amounts of metadata. supports Iceberg 1.6.1.

Snapshot 84

Snapshot Metadata Data Lake Optimization 84

O'Reilly on Data

JUNE 14, 2024

This new service is a trustworthy source of answers for the O’Reilly learning community and a new step forward in the company’s commitment to the experts and authors who drive knowledge across its learning platform. It is possible. This isn’t just a theory; it’s a solution born from direct applied practice.

Metadata 298

Metadata Publishing Modeling Data-driven 298

AWS Big Data

NOVEMBER 20, 2024

We’re excited to announce a new feature in Amazon DataZone that offers enhanced metadata governance for your subscription approval process. With this update, domain owners can define and enforce metadata requirements for data consumers when they request access to data assets. Key benefits The feature benefits multiple stakeholders.

Metadata 76

Metadata Data Governance Metrics Marketing 76

Rocket-Powered Data Science

JULY 6, 2021

This is accomplished through tags, annotations, and metadata (TAM). Smart content includes labeled (tagged, annotated) metadata (TAM). Do you present your employees with a present for their innovative ideas? Do you perfect your plans in anticipation of perfect outcomes? If you have good answers to these questions, that is awesome!

Strategy 267

Strategy Machine Learning Metadata Knowledge Discovery 267

AWS Big Data

NOVEMBER 11, 2024

You can use this approach for a variety of use cases, from real-time log analytics to integrating application messaging data for real-time search. In this post, we focus on the use case for centralizing log aggregation for an organization that has a compliance need to archive and retain its log data.

Metadata 109

Metadata Metrics Analytics Data Processing 109

AWS Big Data

NOVEMBER 19, 2024

Pricing and availability Amazon MWAA pricing dimensions remains unchanged, and you only pay for what you use: The environment class Metadata database storage consumed Metadata database storage pricing remains the same. It enhances infrastructure security and availability while reducing operational overhead. The introduction of mw1.micro

Metadata 106

Metadata Cost-Benefit Metrics Optimization 106

Cloudera

OCTOBER 23, 2024

In August, we wrote about how in a future where distributed data architectures are inevitable, unifying and managing operational and business metadata is critical to successfully maximizing the value of data, analytics, and AI. It is a critical feature for delivering unified access to data in distributed, multi-engine architectures.

Metadata 84

Metadata Data Lake Dashboards Interactive 84

AWS Big Data

NOVEMBER 19, 2024

Solution overview By combining the powerful vector search capabilities of OpenSearch Service with the access control features provided by Amazon Cognito , this solution enables organizations to manage access controls based on custom user attributes and document metadata. If you don’t already have an AWS account, you can create one.

Management 94

Management Metadata Manufacturing Testing 94

CIO Business Intelligence

NOVEMBER 27, 2024

But along with siloed data and compliance concerns , poor data quality is holding back enterprise AI projects. And while most executives generally trust their data, they also say less than two thirds of it is usable. That requires curation and cleaning for hygiene and consistency, and it also requires a feedback loop.”

Enterprise 141

Enterprise Data Quality Structured Data Modeling 141

Rocket-Powered Data Science

MARCH 10, 2020

The book is awesome, an absolute must-have reference volume, and it is free (for now, downloadable from Neo4j ). Graph Algorithms book. Now, for the first time, the full unabridged (and unedited) version of my initial contribution as the Foreword for the book is published here. The campaign looks like a failure.

Metadata 250

Metadata Machine Learning Prescriptive Analytics ROI 250

CIO Business Intelligence

NOVEMBER 19, 2024

For AI to be effective, the relevant data must be easily discoverable and accessible, which requires powerful metadata management and data exploration tools. An enhanced metadata management engine helps customers understand all the data assets in their organization so that they can simplify model training and fine tuning.

Management 116

Management Unstructured Data Deep Learning Metadata 116

AWS Big Data

OCTOBER 29, 2024

The CDH is used to create, discover, and consume data products through a central metadata catalog, while enforcing permission policies and tightly integrating data engineering, analytics, and machine learning services to streamline the user journey from data to insight. This led to inefficiencies in data governance and access control.

Data Lake 89

Data Lake Sales Metadata Machine Learning 89

CIO Business Intelligence

DECEMBER 21, 2022

Every day, organizations of every description are deluged with data from a variety of sources, and attempting to make sense of it all can be overwhelming. So a strong business intelligence (BI) strategy can help organize the flow and ensure business users have access to actionable business insights. “By

IT 131

IT Business Intelligence Sales Key Performance Indicator 131

CIO Business Intelligence

APRIL 29, 2022

As someone who is passionate about the transformative power of technology, it is fascinating to see intelligent computing – in all its various guises – bridge the schism between fantasy and reality. Organisations the world over are in the process of establishing where and how these advancements can add value and edge them closer to their goals.

Data Governance 145

Data Governance IT Risk Data Lake 145

CIO Business Intelligence

SEPTEMBER 26, 2023

But even as we remember 2023 as the year when generative AI went ballistic, AI and its ML (machine learning) sidekick have been quietly evolving over several years to yield eye-opening insights and problem-solving productivity for IT organizations. It’s called AIOps, Artificial Intelligence for IT Operations: next-generation IT management software.

IT 138

IT Key Performance Indicator Software Metadata 138

AWS Big Data

NOVEMBER 6, 2024

reduces the Amazon DynamoDB cost associated with KCL by optimizing read operations on the DynamoDB table storing metadata. KCL uses DynamoDB to store metadata such as shard-worker mapping and checkpoints. and why it results in higher costs. In the sample workload, the producer application publishes 2.5MBps of data across four shards.

Cost-Benefit 109

Cost-Benefit Metadata Optimization Publishing 109