Alation

AUGUST 18, 2022

The complex challenge here is to have the lineage be intelligently updated as the data landscape and processing dynamically bubbles and changes daily across an enterprise. Active metadata will play a critical role in automating such updates as they arise. Get the latest data cataloging news and trends in your inbox.

Metadata 69

Metadata Visualization Statistics Data Architecture 69

AWS Big Data

JUNE 9, 2023

A modern data architecture enables companies to ingest virtually any type of data through automated pipelines into a data lake, which provides highly durable and cost-effective object storage at petabyte or exabyte scale.

Data Lake 125

Data Lake Metadata Statistics Optimization 125

AWS Big Data

DECEMBER 19, 2024

The business end-users were given a tool to discover data assets produced within the mesh and seamlessly self-serve on their data sharing needs. The integration of Databricks Delta tables into Amazon DataZone is done using the AWS Glue Data Catalog. Oghosa Omorisiagbon is a Senior Data Engineer at HEMA.

Data Governance 52

Data Governance Publishing Data-driven Metadata 52

AWS Big Data

AUGUST 31, 2023

Iceberg stores the metadata pointer for all the metadata files. When a SELECT query is reading an Iceberg table, the query engine first goes to the Iceberg catalog, then retrieves the entry of the location of the latest metadata file, as shown in the following diagram.

Data Lake 107

Data Lake Data Warehouse Metadata Data Architecture 107

AWS Big Data

NOVEMBER 29, 2023

Use one click to access your data lake tables using auto-mounted AWS Glue data catalogs on Amazon Redshift for a simplified experience. Learn more about the zero-ETL integrations, data lake performance enhancements, and other announcements below.

Data Warehouse 112

Data Warehouse Analytics Data Lake Machine Learning 112

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. The File Manager Lambda function consumes those messages, parses the metadata, and inserts the metadata to the DynamoDB table odpf_file_tracker.

Data Lake 103

Data Lake Data Processing Metadata Snapshot 103

Cloudera

SEPTEMBER 15, 2022

The consumption of the data should be supported through an elastic delivery layer that aligns with demand, but also provides the flexibility to present the data in a physical format that aligns with the analytic application, ranging from the more traditional data warehouse view to a graph view in support of relationship analysis.

Data Lake 83

Data Lake Data Architecture Data-driven Data Warehouse 83

Alation

JULY 7, 2022

In 2013 I joined American Family Insurance as a metadata analyst. I had always been fascinated by how people find, organize, and access information, so a metadata management role after school was a natural choice. The use cases for metadata are boundless, offering opportunities for innovation in every sector.

Metadata 52

Metadata Data-driven Insurance Statistics 52

Alation

JANUARY 25, 2022

As a reminder, here’s Gartner’s definition of data fabric: “A design concept that serves as an integrated layer (fabric) of data and connecting processes. In this blog, we will focus on the “integrated layer” part of this definition by examining each of the key layers of a comprehensive data fabric in more detail.

Metadata 52

Metadata IT Data-driven Metrics 52

AWS Big Data

APRIL 3, 2024

If the asset has AWS Glue Data Quality enabled, you can now quickly visualize the data quality score directly in the catalog search pane. By selecting the corresponding asset, you can understand its content through the readme, glossary terms , and technical and business metadata.

Data Quality 103

Data Quality Visualization Metadata Metrics 103

Sisense

SEPTEMBER 3, 2019

Modeling Your Data for Performance. Data architecture. The data landscape has changed significantly over the last two decades. The volume of data being created has increased, and the storage and computational resources needed to store and analyze that data has become cheaper and more widely available.

Data Warehouse 78

Data Warehouse OLAP Statistics Cost-Benefit 78

Alation

APRIL 25, 2023

But whatever your industry, perfecting your processes for making important decisions about how to handle data is crucial. Whether you deal in customer contact information, website traffic statistics, sales data, or some other type of valuable information, you’ll need to put a framework of policies in place to manage your data seamlessly.

Data Governance 52

Data Governance Marketing Machine Learning Sales 52

Alation

APRIL 25, 2023

But whatever your industry, perfecting your processes for making important decisions about how to handle data is crucial. Whether you deal in customer contact information, website traffic statistics, sales data, or some other type of valuable information, you’ll need to put a framework of policies in place to manage your data seamlessly.

Data Governance 40

Data Governance Marketing Machine Learning Sales 40

Andrew White

JANUARY 9, 2022

We found anecdotal data that suggested things such as a) CDO’s with a business, more than a technical, background tend to be more effective or successful, and b) CDOs most often came from a business background, and c) those that were successful had a good chance at becoming CEO or CEO or some other CXO (but not really CIO).

Analytics 64

Analytics Measurement Data-driven Modeling 64

Peter James Thomas

AUGUST 18, 2018

Even back then, these were used for activities such as Analytics , Dashboards , Statistical Modelling , Data Mining and Advanced Visualisation. Next, rather than just being the province of Data Scientists, there were moves to use Data Lakes to support general Data Discovery and even business Reporting and Analytics as well.

Data Lake 63

Data Lake Data Warehouse Data mining Statistics 63

Jet Global

MAY 1, 2023

Some cloud applications can even provide new benchmarks based on customer data. Advanced Analytics Some apps provide a unique value proposition through the development of advanced (and often proprietary) statistical models. Advanced Analytics Provide the unique benefit of advanced (and often proprietary) statistical models in your app.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52

Ontotext

NOVEMBER 11, 2024

Knowledge graphs, while not as well-known as other data management offerings, are a proven dynamic and scalable solution for addressing enterprise data management requirements across several verticals. The RDF-star extension makes it easy to model provenance and other structured metadata.

Metadata 52

Metadata Knowledge Discovery Data Integration Management 52

DataKitchen

JANUARY 23, 2023

Data testing is an essential aspect of DataOps Observability; it helps to ensure that data is accurate, complete, and consistent with its specifications, documentation, and end-user requirements. Data testing can be done through various methods, such as data profiling, Statistical Process Control, and quality checks.

Testing 130

Testing Data Quality Data Governance Data-driven 130