erwin

APRIL 29, 2020

Data lineage is the journey data takes from its creation through its transformations over time. Tracing the source of data is an arduous task. With all these diverse data sources, and if systems are integrated, it is difficult to understand the complicated data web they form much less get a simple visual flow.

Metadata 111

Metadata Key Performance Indicator Data Governance Data Quality 111

Wayne Yaddow

FEBRUARY 18, 2025

Selecting the strategies and tools for validating data transformations and data conversions in your data pipelines. Introduction Data transformations and data conversions are crucial to ensure that raw data is organized, processed, and ready for useful analysis.

Testing 52

Testing Data Transformation Data-driven Data Quality 52

AWS Big Data

OCTOBER 11, 2023

Organizations with legacy, on-premises, near-real-time analytics solutions typically rely on self-managed relational databases as their data store for analytics workloads. Near-real-time streaming analytics captures the value of operational data and metrics to provide new insights to create business opportunities.

Management 117

Management Metadata Analytics Dashboards 117

Alation

JUNE 28, 2022

Collecting and using data to make informed decisions is the new foundation for businesses. The key term here is usable : Anyone can be data rich, and collect vast troves of data. This is where metadata, or the data about data, comes into play. What is a Metadata Management Framework?

Metadata 52

Metadata Management Data Governance Machine Learning 52

AWS Big Data

DECEMBER 13, 2023

In addition to using native managed AWS services that BMS didn’t need to worry about upgrading, BMS was looking to offer an ETL service to non-technical business users that could visually compose data transformation workflows and seamlessly run them on the AWS Glue Apache Spark-based serverless data integration engine.

Metadata 100

Metadata Data Lake Visualization Data Quality 100

Wayne Yaddow

MARCH 14, 2025

How dbt Core aids data teams test, validate, and monitor complex data transformations and conversions Photo by NASA on Unsplash Introduction dbt Core, an open-source framework for developing, testing, and documenting SQL-based data transformations, has become a must-have tool for modern data teams as the complexity of data pipelines grows.

Data Transformation 52

Data Transformation Testing Unstructured Data Data Quality 52

erwin

SEPTEMBER 17, 2020

Replace manual and recurring tasks for fast, reliable data lineage and overall data governance. It’s paramount that organizations understand the benefits of automating end-to-end data lineage. The importance of end-to-end data lineage is widely understood and ignoring it is risky business. Doing Data Lineage Right.

Cost-Benefit 143

Cost-Benefit Data Governance Metadata Reporting 143

AWS Big Data

DECEMBER 16, 2024

The need to integrate diverse data sources has grown exponentially, but there are several common challenges when integrating and analyzing data from multiple sources, services, and applications. First, you need to create and maintain independent connections to the same data source for different services.

Visualization 71

Visualization Data Processing Testing Publishing 71

CIO Business Intelligence

AUGUST 3, 2023

When global technology company Lenovo started utilizing data analytics, they helped identify a new market niche for its gaming laptops, and powered remote diagnostics so their customers got the most from their servers and other devices.

Analytics 98

Analytics Data Lake Metadata Cost-Benefit 98

Ontotext

FEBRUARY 12, 2024

In the final part of this three-part series, we’ll explore ho w data mesh bolsters performance and helps organizations and data teams work more effectively. Usually, organizations will combine different domain topologies, depending on the trade-offs, and choose to focus on specific aspects of data mesh.

Data-driven 59

Data-driven Data Lake Data Quality Business Objectives 59

erwin

MARCH 4, 2021

Modern data governance is a strategic, ongoing and collaborative practice that enables organizations to discover and track their data, understand what it means within a business context, and maximize its security, quality and value. The What: Data Governance Defined. Data governance has no standard definition.

Data Governance 122

Data Governance Digital Transformation Data-driven Cost-Benefit 122

AWS Big Data

NOVEMBER 15, 2023

It seamlessly consolidates data from various data sources within AWS, including AWS Cost Explorer (and forecasting with Cost Explorer ), AWS Trusted Advisor , and AWS Compute Optimizer. Overview of the BMW Cloud Data Hub At the BMW Group, Cloud Data Hub (CDH) is the central platform for managing company-wide data and data solutions.

Dashboards 106

Dashboards Analytics Metadata Data Warehouse 106

CIO Business Intelligence

FEBRUARY 20, 2024

Manufacturers have long held a data-driven vision for the future of their industry. It’s one where near real-time data flows seamlessly between IT and operational technology (OT) systems. Legacy data management is holding back manufacturing transformation Until now, however, this vision has remained out of reach.

Data Architecture 98

Data Architecture Unstructured Data Manufacturing IT 98

Alation

OCTOBER 26, 2021

Today’s best-performing organizations embrace data for strategic decision-making. Because of the criticality of the data they deal with, we think that finance teams should lead the enterprise adoption of data and analytics solutions. This is because accurate data is “table stakes” for finance teams.

Finance 52

Finance Data Transformation Enterprise Metrics 52

Cloudera

JUNE 30, 2022

We are excited to announce the general availability of Apache Iceberg in Cloudera Data Platform (CDP). These tools empower analysts and data scientists to easily collaborate on the same data, with their choice of tools and analytic engines. Why integrate Apache Iceberg with Cloudera Data Platform?

Data Lake 88

Data Lake Data Warehouse Data Architecture Metadata 88

Cloudera

OCTOBER 11, 2021

Modak, a leading provider of modern data engineering solutions, is now a certified solution partner with Cloudera. Customers can now seamlessly automate migration to Cloudera’s Hybrid Data Platform — Cloudera Data Platform (CDP) to dynamically auto-scale cloud services with Cloudera Data Engineering (CDE) integration with Modak Nabu.

Data Lake 95

Data Lake Cost-Benefit Data-driven Dashboards 95

Cloudera

OCTOBER 7, 2022

dbt allows data teams to produce trusted data sets for reporting, ML modeling, and operational workflows using SQL, with a simple workflow that follows software engineering best practices like modularity, portability, and continuous integration/continuous development (CI/CD). The Open Data Lakehouse . Introduction.

Data Warehouse 81

Data Warehouse Data Transformation Machine Learning Data Lake 81

AWS Big Data

DECEMBER 21, 2023

This is a guest post co-written by Alex Naumov, Principal Data Architect at smava. smava believes in and takes advantage of data-driven decisions in order to become the market leader. smava believes in and takes advantage of data-driven decisions in order to become the market leader.

Data Lake 103

Data Lake Data Warehouse Data-driven B2B 103

AWS Big Data

OCTOBER 11, 2023

Under the Transparency in Coverage (TCR) rule , hospitals and payors to publish their pricing data in a machine-readable format. The data in the machine-readable files can provide valuable insights to understand the true cost of healthcare services and compare prices and quality across hospitals.

Visualization 110

Visualization Dashboards Data-driven Gap analysis 110

AWS Big Data

APRIL 19, 2023

Data mesh is a new approach to data management. Companies across industries are using a data mesh to decentralize data management to improve data agility and get value from data. This is especially true in a large enterprise with thousands of data products.

Technology 92

Technology Data-driven Machine Learning Sales 92

AWS Big Data

JANUARY 27, 2023

To build a data-driven business, it is important to democratize enterprise data assets in a data catalog. With a unified data catalog, you can quickly search datasets and figure out data schema, data format, and location. For metadata read/write, Flink has the catalog interface.

Data Lake 66

Data Lake Metadata Business Analysis Data-driven 66

erwin

JANUARY 11, 2019

Organizations have spent a lot of time and money trying to harmonize data across diverse platforms , including cleansing, uploading metadata, converting code, defining business glossaries, tracking data transformations and so on. So questions linger about whether transformed data can be trusted.

Data Governance 40

Data Governance Risk Metadata Management 40

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

Data Lake 93

Data Lake Analytics Snapshot Data Quality 93

Alation

NOVEMBER 22, 2022

What Is Data Governance In The Public Sector? Effective data governance for the public sector enables entities to ensure data quality, enhance security, protect privacy, and meet compliance requirements. With so much focus on compliance, democratizing data for self-service analytics can present a challenge.

Data Governance 52

Data Governance Metadata Data-driven Unstructured Data 52

AWS Big Data

AUGUST 1, 2023

Although Jira Cloud provides reporting capability, loading this data into a data lake will facilitate enrichment with other business data, as well as support the use of business intelligence (BI) tools and artificial intelligence (AI) and machine learning (ML) applications.

Data Lake 92

Data Lake Data Transformation Data-driven Cost-Benefit 92

Octopai

JUNE 9, 2024

In today’s data-centric world, organizations often tout data as their most valuable asset. However, many struggle to maintain reliable, trustworthy data amidst complex, evolving environments. This challenge is especially critical for executives responsible for data strategy and operations.

IT 52

IT Data-driven Predictive Analytics Data Strategy 52

Cloudera

MARCH 14, 2023

We just announced the general availability of Cloudera DataFlow Designer , bringing self-service data flow development to all CDP Public Cloud customers. In this blog post we will put these capabilities in context and dive deeper into how the built-in, end-to-end data flow life cycle enables self-service data pipeline development.

Testing 84

Testing Publishing Metadata Interactive 84

Octopai

AUGUST 11, 2024

Data flow lineage is crucial for anyone handling data within organizations. In essence, data flow lineage is indispensable for ensuring transparency, maintaining data quality, achieving compliance, enabling efficient troubleshooting, conducting impact analysis, and enhancing collaboration within organizations.

Metadata 40

Metadata Visualization Data Quality Data Governance 40

IBM Big Data Hub

JANUARY 18, 2023

Data platform architecture has an interesting history. A read-optimized platform that can integrate data from multiple applications emerged. In another decade, the internet and mobile started the generate data of unforeseen volume, variety and velocity. It required a different data platform solution. It is too expensive.

Data Lake 52

Data Lake Data Warehouse Data-driven Metadata 52

Alation

MARCH 22, 2022

Data people face a challenge. They must put high-quality data into the hands of users as efficiently as possible. As the latest iteration in this pursuit of high-quality data sharing, DataOps combines a range of disciplines. It synthesizes all we’ve learned about agile, data quality , and ETL/ELT.

Metadata 52

Metadata Cost-Benefit Data Quality Data Lake 52

AWS Big Data

MARCH 15, 2023

Infomedia Ltd (ASX:IFM) is a leading global provider of DaaS and SaaS solutions that empowers the data-driven automotive ecosystem. In this post, we share how Infomedia built a serverless data pipeline with change data capture (CDC) using AWS Glue and Apache Hudi.

Cost-Benefit 72

Cost-Benefit Data Processing Optimization Data-driven 72

Ontotext

MARCH 27, 2024

For example, GPS, social media, cell phone handoffs are modeled as graphs while data catalogs, data lineage and MDM tools leverage knowledge graphs for linking metadata with semantics. Both nodes and edges have associated properties modeled as key-values with primitive data types and are single-valued.

Modeling 53

Modeling Metadata Data Quality Enterprise 53

AWS Big Data

JANUARY 17, 2024

This allows you to simplify security and governance over transactional data lakes by providing access controls at table-, column-, and row-level permissions with your Apache Spark jobs. Many large enterprise companies seek to use their transactional data lake to gain insights and improve decision-making.

Data Lake 107

Data Lake Snapshot Big Data Data-driven 107

Alation

JANUARY 17, 2023

Chances are, you’ve heard of the term “modern data stack” before. In this article, I will explain the modern data stack in detail, list some benefits, and discuss what the future holds. What Is the Modern Data Stack? It is known to have benefits in handling data due to its robustness, speed, and scalability.

Data Warehouse 52

Data Warehouse Cost-Benefit Data Science Data Transformation 52

AWS Big Data

APRIL 25, 2024

In the era of data, organizations are increasingly using data lakes to store and analyze vast amounts of structured and unstructured data. Data lakes provide a centralized repository for data from various sources, enabling organizations to unlock valuable insights and drive data-driven decision-making.

Optimization 72

Optimization Data Lake Cost-Benefit Reporting 72

Jet Global

FEBRUARY 23, 2024

The quick and dirty definition of data mapping is the process of connecting different types of data from various data sources. Data mapping is a crucial step in data modeling and can help organizations achieve their business goals by enabling data integration, migration, transformation, and quality.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Visualization 52

Jet Global

MAY 1, 2023

By leveraging data analysis to solve high-value business problems, they will become more efficient. This is in contrast to traditional BI, which extracts insight from data outside of the app. that gathers data from many sources. These tools prep that data for analysis and then provide reporting on it from a central viewpoint.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52