article thumbnail

The Race For Data Quality in a Medallion Architecture

DataKitchen

The Race For Data Quality In A Medallion Architecture The Medallion architecture pattern is gaining traction among data teams. It is a layered approach to managing and transforming data. It sounds great, but how do you prove the data is correct at each layer? How do you ensure data quality in every layer ?

article thumbnail

Automating Data Quality Checks with Dagster and Great Expectations

Analytics Vidhya

Introduction Ensuring data quality is paramount for businesses relying on data-driven decision-making. As data volumes grow and sources diversify, manual quality checks become increasingly impractical and error-prone.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Build Write-Audit-Publish pattern with Apache Iceberg branching and AWS Glue Data Quality

AWS Big Data

Equally crucial is the ability to segregate and audit problematic data, not just for maintaining data integrity, but also for regulatory compliance, error analysis, and potential data recovery. We discuss two common strategies to verify the quality of published data.

article thumbnail

Data Observability and Data Quality Testing Certification Series

DataKitchen

Data Observability and Data Quality Testing Certification Series We are excited to invite you to a free four-part webinar series that will elevate your understanding and skills in Data Observation and Data Quality Testing. Slides and recordings will be provided.

article thumbnail

Simplify data integration with AWS Glue and zero-ETL to Amazon SageMaker Lakehouse

AWS Big Data

With the growing emphasis on data, organizations are constantly seeking more efficient and agile ways to integrate their data, especially from a wide variety of applications. We take care of the ETL for you by automating the creation and management of data replication. What’s the difference between zero-ETL and Glue ETL?

article thumbnail

Data Quality Is Free

Anmut

They made us realise that building systems, processes and procedures to ensure quality is built in at the outset is far more cost effective than correcting mistakes once made. How about data quality? Redman and David Sammon, propose an interesting (and simple) exercise to measure data quality.

article thumbnail

Talend Data Fabric Simplifies Data Life Cycle Management

David Menninger's Analyst Perspectives

Talend is a data integration and management software company that offers applications for cloud computing, big data integration, application integration, data quality and master data management.