Remove automated-testing-and-monitoring
article thumbnail

The Race For Data Quality in a Medallion Architecture

DataKitchen

The Race For Data Quality In A Medallion Architecture The Medallion architecture pattern is gaining traction among data teams. It is a layered approach to managing and transforming data. By systematically moving data through these layers, the Medallion architecture enhances the data structure in a data lakehouse environment.

article thumbnail

Announcing Open Source DataOps Data Quality TestGen 3.0

DataKitchen

Now With Actionable, Automatic, Data Quality Dashboards Imagine a tool that can point at any dataset, learn from your data, screen for typical data quality issues, and then automatically generate and perform powerful tests, analyzing and scoring your data to pinpoint issues before they snowball. New Quality Dashboard & Score Explorer.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Agentic AI design: An architectural case study

CIO Business Intelligence

Agents will play different roles as part of a complex workflow, automating tasks more efficiently. They can handle complex tasks, including planning, reasoning, learning from experience, and automating activities to achieve their goal. There are many areas of research and focus sprouting from the capabilities presented through LLMs.

Testing 135
article thumbnail

The DataOps Vendor Landscape, 2021

DataKitchen

Testing and Data Observability. It orchestrates complex pipelines, toolchains, and tests across teams, locations, and data centers. Airflow — An open-source platform to programmatically author, schedule, and monitor data pipelines. Download the 2021 DataOps Vendor Landscape here. DataOps is a hot topic in 2021.

Testing 300
article thumbnail

Escaping POC Purgatory: Evaluation-Driven Development for AI Systems

O'Reilly on Data

Weve seen this across dozens of companies, and the teams that break out of this trap all adopt some version of Evaluation-Driven Development (EDD), where testing, monitoring, and evaluation drive every decision from the start. Lets be real: building LLM applications today feels like purgatory. Leadership gets excited. The way out?

Testing 174
article thumbnail

Bigeye Enable Monitoring, Quality and Lineage of Data

David Menninger's Analyst Perspectives

Data observability addresses one of the most significant impediments to generating value from data by providing an environment for monitoring the quality and reliability of data on a continual basis. Bigeye was founded in late 2018 by Chief Executive Officer Kyle Kirwan and Chief Technology Officer Egor Gryaznov. The company has raised $73.5

article thumbnail

Introducing simplified interaction with the Airflow REST API in Amazon MWAA

AWS Big Data

The Airflow REST API facilitates a wide range of use cases, from centralizing and automating administrative tasks to building event-driven, data-aware data pipelines. Today, we are excited to announce an enhancement to the Amazon MWAA integration with the Airflow REST API.