erwin

FEBRUARY 7, 2019

Organization’s cannot hope to make the most out of a data-driven strategy, without at least some degree of metadata-driven automation. The volume and variety of data has snowballed, and so has its velocity. As such, traditional – and mostly manual – processes associated with data management and data governance have broken down.

Metadata 96

Metadata Insurance Data-driven Cost-Benefit 96

Cloudera

MAY 17, 2021

Making a decision on a cloud data warehouse is a big deal. Modernizing your data warehousing experience with the cloud means moving from dedicated, on-premises hardware focused on traditional relational analytics on structured data to a modern platform.

Data Warehouse 92

Data Warehouse Measurement Reporting Testing 92

datapine

MAY 29, 2019

One of the BI architecture components is data warehousing. Organizing, storing, cleaning, and extraction of the data must be carried by a central repository system, namely data warehouse, that is considered as the fundamental component of business intelligence. What Is Data Warehousing And Business Intelligence?

Business Intelligence 196

Business Intelligence Data Warehouse Dashboards Visualization 196

DataKitchen

AUGUST 3, 2021

The past decades of enterprise data platform architectures can be summarized in 69 words. First-generation – expensive, proprietary enterprise data warehouse and business intelligence platforms maintained by a specialized team drowning in technical debt. The post What is a Data Mesh? first appeared on DataKitchen.

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

erwin

DECEMBER 12, 2019

Metadata is an important part of data governance, and as a result, most nascent data governance programs are rife with project plans for assessing and documenting metadata. But in many scenarios, it seems that the underlying driver of metadata collection projects is that it’s just something you do for data governance.

Metadata 84

Metadata Data Governance Digital Transformation Data Quality 84

Cloudera

APRIL 3, 2023

In this blog, we will share with you in detail how Cloudera integrates core compute engines including Apache Hive and Apache Impala in Cloudera Data Warehouse with Iceberg. We will publish follow up blogs for other data services. Iceberg basics Iceberg is an open table format designed for large analytic workloads.

Data Warehouse 87

Data Warehouse Snapshot Metadata Cost-Benefit 87

Cloudera

SEPTEMBER 24, 2020

Some of the most powerful results come from combining complementary superpowers, and the “dynamic duo” of Apache Hive LLAP and Apache Impala, both included in Cloudera Data Warehouse , is further evidence of this. Both Impala and Hive can operate at an unprecedented and massive scale, with many petabytes of data.

Data Warehouse 75

Data Warehouse Metadata Interactive Dashboards 75

Alation

JULY 28, 2022

It’s costly and time-consuming to manage on-premises data warehouses — and modern cloud data architectures can deliver business agility and innovation. However, CIOs declare that agility, innovation, security, adopting new capabilities, and time to value — never cost — are the top drivers for cloud data warehousing.

Data Warehouse 69

Data Warehouse Cost-Benefit Data-driven Data Governance 69

Data Virtualization

DECEMBER 8, 2022

Reading Time: 3 minutes While cleaning up our archive recently, I found an old article published in 1976 about data dictionary/directory systems (DD/DS). Nowadays, we no longer use the term DD/DS, but “data catalog” or simply “metadata system”. It was written by L.

Metadata 69

Metadata IT Data Integration Publishing 69

Cloudera

FEBRUARY 9, 2021

Today’s customers have a growing need for a faster end to end data ingestion to meet the expected speed of insights and overall business demand. This ‘need for speed’ drives a rethink on building a more modern data warehouse solution, one that balances speed with platform cost management, performance, and reliability.

Data Warehouse 73

Data Warehouse Cost-Benefit Metadata Management 73

erwin

JUNE 26, 2020

Given the value this sort of data-driven insight can provide, the reason organizations need a data catalog should become clearer. It’s no surprise that most organizations’ data is often fragmented and siloed across numerous sources (e.g., Three Types of Metadata in a Data Catalog. Technical Metadata.

Metadata 132

Metadata Cost-Benefit Measurement Data-driven 132

Timo Elliott

JANUARY 6, 2022

Like the proverbial man looking for his keys under the streetlight , when it comes to enterprise data, if you only look at where the light is already shining, you can end up missing a lot. The data you’ve collected and saved over the years isn’t free. Analyze your metadata. Data sense-making. Storing data isn’t enough.

IT 135

IT Metadata Data-driven Data Governance 135

AWS Big Data

NOVEMBER 14, 2024

You can learn how to query Delta Lake native tables through UniForm from different data warehouses or engines such as Amazon Redshift as an example of expanding data access to more engines. Both Delta Lake and Iceberg metadata files reference the same data files.

Metadata 121

Metadata Data Warehouse Big Data Data Lake 121

AWS Big Data

DECEMBER 17, 2024

Amazon Redshift , launched in 2013, has undergone significant evolution since its inception, allowing customers to expand the horizons of data warehousing and SQL analytics. Industry-leading price-performance Amazon Redshift offers up to three times better price-performance than alternative cloud data warehouses.

Data Lake 78

Data Lake Data Warehouse Data-driven Optimization 78

Data Virtualization

APRIL 21, 2022

Reading Time: 3 minutes First we had data warehouses, then came data lakes, and now the new kid on the block is the data lakehouse. But what is a data lakehouse and why should we develop one? In a way, the name describes what.

Data Lake 59

Data Lake Data Warehouse Data Integration Management 59

Sisense

SEPTEMBER 3, 2019

This blog is intended to give an overview of the considerations you’ll want to make as you build your Redshift data warehouse to ensure you are getting the optimal performance. This results in less joins between the metric data in fact tables, and the dimensions. So let’s dive in! OLTP vs OLAP.

Data Warehouse 78

Data Warehouse OLAP Statistics Cost-Benefit 78

Cloudera

FEBRUARY 8, 2021

This is part 2 in this blog series. You can read part 1, here: Digital Transformation is a Data Journey From Edge to Insight. The first blog introduced a mock connected vehicle manufacturing company, The Electric Car Company (ECC), to illustrate the manufacturing data path through the data lifecycle.

Manufacturing 122

Manufacturing Data Warehouse Sales Predictive Analytics 122

Cloudera

DECEMBER 10, 2020

Why worry about costs with cloud-native data warehousing? Have you been burned by the unexpected costs of a cloud data warehouse? If not, before adopting a cloud data warehouse, consider the true costs of a cloud-native data warehouse. These costs impede the adoption of cloud-native data warehouses.

Data Warehouse 98

Data Warehouse Metadata Cost-Benefit Optimization 98

AWS Big Data

MARCH 2, 2023

Apache Iceberg is an open table format for very large analytic datasets, which captures metadata information on the state of datasets as they evolve and change over time. Iceberg has become very popular for its support for ACID transactions in data lakes and features like schema and partition evolution, time travel, and rollback.

Data Lake 136

Data Lake Data Processing Metadata Snapshot 136

Cloudera

SEPTEMBER 9, 2018

We are proud to announce the general availability of Cloudera Altus Data Warehouse , the only cloud data warehousing service that brings the warehouse to the data. Modern data warehousing for the cloud. Modern data warehousing for the cloud. Using Cloudera Altus for your cloud data warehouse.

Data Warehouse 40

Data Warehouse Metadata Cost-Benefit Reporting 40

erwin

DECEMBER 12, 2019

Metadata is an important part of data governance, and as a result, most nascent data governance programs are rife with project plans for assessing and documenting metadata. But in many scenarios, it seems that the underlying driver of metadata collection projects is that it’s just something you do for data governance.

Metadata 52

Metadata Data Governance Digital Transformation Data Quality 52

Cloudera

FEBRUARY 8, 2024

Overview This blog post describes support for materialized views for the Iceberg table format. It brings the reliability and simplicity of SQL tables to big data while enabling engines like Hive, Impala, Spark, Trino, Flink, and Presto to work with the same tables at the same time. Starting from the CDW Public Cloud DWX-1.6.1

Snapshot 96

Snapshot Metadata Cost-Benefit Data Warehouse 96

AWS Big Data

JUNE 10, 2024

In this blog post, we dive into different data aspects and how Cloudinary breaks the two concerns of vendor locking and cost efficient data analytics by using Apache Iceberg, Amazon Simple Storage Service (Amazon S3 ), Amazon Athena , Amazon EMR , and AWS Glue. Old metadata files are kept for history by default.

Data Lake 126

Data Lake Metadata Snapshot Analytics 126

AWS Big Data

SEPTEMBER 11, 2024

This blog post is co-written with Hardeep Randhawa and Abhay Kumar from HPE. The data sources include 150+ files including 10-15 mandatory files per region ingested in various formats like xlxs, csv, and dat. Each file arrives as a pair with a tail metadata file in CSV format containing the size and name of the file.

Data Architecture 110

Data Architecture Optimization Data Warehouse Metadata 110

AWS Big Data

AUGUST 31, 2023

Amazon Redshift is a fast, fully managed petabyte-scale cloud data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing business intelligence (BI) tools. Amazon Redshift also supports querying nested data with complex data types such as struct, array, and map.

Data Lake 118

Data Lake Data Warehouse Metadata Data Architecture 118

Cloudera

AUGUST 31, 2021

Cloudera and Accenture demonstrate strength in their relationship with an accelerator called the Smart Data Transition Toolkit for migration of legacy data warehouses into Cloudera Data Platform. Accenture’s Smart Data Transition Toolkit . Are you looking for your data warehouse to support the hybrid multi-cloud?

Data Warehouse 87

Data Warehouse Cost-Benefit Metadata Data-driven 87

Alation

FEBRUARY 13, 2020

In an earlier blog, I defined a data catalog as “a collection of metadata, combined with data management and search tools, that helps analysts and other data users to find the data that they need, serves as an inventory of available data, and provides information to evaluate fitness data for intended uses.”.

Metadata 52

Metadata Management Data Lake Data Governance 52

Cloudera

JUNE 18, 2022

These lakes power mission critical large scale data analytics, business intelligence (BI), and machine learning use cases, including enterprise data warehouses. In recent years, the term “data lakehouse” was coined to describe this architectural pattern of tabular analytics over data in the data lake.

Data Lake 98

Data Lake Data Warehouse Machine Learning Cost-Benefit 98

Cloudera

NOVEMBER 5, 2020

Many organizations struggle to meet growing and variable data warehouse demands. This is exactly what Cloudera Data Platform (CDP) provides to the Cloudera Data Warehouse. CDP is a data platform that is optimized for both business units and central IT. . Cloudera Data Warehouse Security.

Data Lake 76

Data Lake Data Warehouse Metadata Optimization 76

Ontotext

DECEMBER 15, 2022

But whatever their business goals, in order to turn their invisible data into a valuable asset, they need to understand what they have and to be able to efficiently find what they need. Enter metadata. It enables us to make sense of our data because it tells us what it is and how best to use it. Knowledge (metadata) layer.

Metadata 52

Metadata Unstructured Data Structured Data Enterprise 52

AWS Big Data

OCTOBER 4, 2023

Additionally, data owners and data stewards can make data discovery simpler by adding business context to data while balancing access governance to the data via pre-defined approval workflows in the user interface. The metadata forms types, and asset types can be used as templates for defining your assets.

Metadata 119

Metadata Data Lake Publishing Data Governance 119

Cloudera

DECEMBER 3, 2024

REST Catalog Value Proposition It provides open, metastore-agnostic APIs for Iceberg metadata operations, dramatically simplifying the Iceberg client and metastore/engine integration. It provides real time metadata access by directly integrating with the Iceberg-compatible metastore. You will see the 2 carrier records in the table.

Metadata 89

Metadata Data Warehouse ROI Snapshot 89

Cloudera

JANUARY 21, 2021

While cloud-native, point-solution data warehouse services may serve your immediate business needs, there are dangers to the corporation as a whole when you do your own IT this way. Cloudera Data Warehouse (CDW) is here to save the day! CDW is an integrated data warehouse service within Cloudera Data Platform (CDP).

Data Warehouse 96

Data Warehouse Data Lake IT Analytics 96

AWS Big Data

DECEMBER 4, 2024

Amazon SageMaker Lakehouse , now generally available, unifies all your data across Amazon Simple Storage Service (Amazon S3) data lakes and Amazon Redshift data warehouses, helping you build powerful analytics and AI/ML applications on a single copy of data. Having confidence in your data is key.

Data Analytics 101

Data Analytics Analytics Data Lake Data Quality 101

Cloudera

JANUARY 19, 2022

Cloudera Data Platform (CDP) scored among the top 10 vendors on all four Analytical Use Cases — Data Warehouse, Logical Data Warehouse, Data Lake and Operational Intelligence in the Critical Capabilities for Cloud Database Management Systems for Analytics Use Cases.

Reporting 96

Reporting Data Warehouse Data Lake Machine Learning 96

erwin

AUGUST 7, 2019

A data catalog benefits organizations in a myriad of ways. With the right data catalog tool, organizations can automate enterprise metadata management – including data cataloging, data mapping, data quality and code generation for faster time to value and greater accuracy for data movement and/or deployment projects.

Metadata 90

Metadata Data Governance Data Quality Data Warehouse 90

Alation

OCTOBER 18, 2022

In the modern data stack, dbt is a key tool to make data ready for analysis. Data analysts and engineers use dbt to transform, test, and document data in the cloud data warehouse. Yet every dbt transformation contains vital metadata that is not captured – until now. Subscribe to Alation's Blog.

Metadata 52

Metadata Metrics Recreation/Entertainment Data Quality 52