erwin

OCTOBER 24, 2019

Untapped data, if mined, represents tremendous potential for your organization. While there has been a lot of talk about big data over the years, the real hero in unlocking the value of enterprise data is metadata , or the data about the data. Metadata Is the Heart of Data Intelligence.

Metadata 104

Metadata Management Data-driven Data Architecture 104

erwin

DECEMBER 12, 2019

Metadata is an important part of data governance, and as a result, most nascent data governance programs are rife with project plans for assessing and documenting metadata. But in many scenarios, it seems that the underlying driver of metadata collection projects is that it’s just something you do for data governance.

Metadata 84

Metadata Data Governance Digital Transformation Data Quality 84

Cloudera

SEPTEMBER 29, 2022

Each of these trends claim to be complete models for their data architectures to solve the “everything everywhere all at once” problem. Data teams are confused as to whether they should get on the bandwagon of just one of these trends or pick a combination. First, we describe how data mesh and data fabric could be related.

Data Architecture 74

Data Architecture Data Warehouse Metadata Sales 74

Cloudera

OCTOBER 23, 2024

In August, we wrote about how in a future where distributed data architectures are inevitable, unifying and managing operational and business metadata is critical to successfully maximizing the value of data, analytics, and AI.

Metadata 83

Metadata Data Lake Dashboards Interactive 83

Alation

AUGUST 18, 2022

How do you approach data lineage? We all know that data lineage is a complex and challenging topic. In this blog, I am drilling into something I’ve been thinking about and studying for a long time: fundamental approaches to lineage creation and maintenance. What Is Data Lineage Creation & Maintenance?

Metadata 69

Metadata Visualization Statistics Data Architecture 69

erwin

DECEMBER 12, 2019

Metadata is an important part of data governance, and as a result, most nascent data governance programs are rife with project plans for assessing and documenting metadata. But in many scenarios, it seems that the underlying driver of metadata collection projects is that it’s just something you do for data governance.

Metadata 52

Metadata Data Governance Digital Transformation Data Quality 52

AWS Big Data

NOVEMBER 14, 2024

The landscape of big data management has been transformed by the rising popularity of open table formats such as Apache Iceberg, Apache Hudi, and Linux Foundation Delta Lake. These formats, designed to address the limitations of traditional data storage systems, have become essential in modern data architectures.

Metadata 98

Metadata Data Warehouse Big Data Data Lake 98

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. The communication between business units and data professionals is usually incomplete and inconsistent. Introduction to Data Mesh. Source: Thoughtworks.

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

TDAN

MAY 31, 2022

The main goal of creating an enterprise data fabric is not new. It is the ability to deliver the right data at the right time, in the right shape, and to the right data consumer, irrespective of how and where it is stored. Data fabric is the common “net” that stitches integrated data from multiple data […].

Metadata 98

Metadata Enterprise IT Data Architecture 98

AWS Big Data

OCTOBER 3, 2023

Over the years, data lakes on Amazon Simple Storage Service (Amazon S3) have become the default repository for enterprise data and are a common choice for a large set of users who query data for a variety of analytics and machine leaning use cases. Analytics use cases on data lakes are always evolving.

Data Lake 110

Data Lake Metadata Snapshot Recreation/Entertainment 110

Cloudera

FEBRUARY 22, 2022

Over the past decade, the successful deployment of large scale data platforms at our customers has acted as a big data flywheel driving demand to bring in even more data, apply more sophisticated analytics, and on-board many new data practitioners from business analysts to data scientists.

Snapshot 109

Snapshot Metadata Cost-Benefit Data Architecture 109

Cloudera

JULY 19, 2023

It is a replicated, highly-available service that is responsible for managing the metadata for all objects stored in Ozone. As Ozone scales to exabytes of data, it is important to ensure that Ozone Manager can perform at scale. The hardware specifications are included at the end of this blog.

Management 70

Management Metadata Metrics Optimization 70

AWS Big Data

JUNE 10, 2024

In this blog post, we dive into different data aspects and how Cloudinary breaks the two concerns of vendor locking and cost efficient data analytics by using Apache Iceberg, Amazon Simple Storage Service (Amazon S3 ), Amazon Athena , Amazon EMR , and AWS Glue. This concept makes Iceberg extremely versatile. SparkActions.get().expireSnapshots(iceTable).expireOlderThan(TimeUnit.DAYS.toMillis(7)).execute()

Data Lake 116

Data Lake Metadata Snapshot Analytics 116

Ontotext

JANUARY 10, 2024

Limiting growth by (data integration) complexity Most operational IT systems in an enterprise have been developed to serve a single business function and they use the simplest possible model for this. A slightly different angle to the same problem is discussed in a recent Ontotext blog post. So, they become very data-driven.

Metadata 72

Metadata Slice and Dice Data Integration Enterprise 72

Ontotext

NOVEMBER 11, 2024

Knowledge graphs, while not as well-known as other data management offerings, are a proven dynamic and scalable solution for addressing enterprise data management requirements across several verticals. The RDF-star extension makes it easy to model provenance and other structured metadata.

Metadata 52

Metadata Knowledge Discovery Data Integration Management 52

IBM Big Data Hub

APRIL 10, 2023

A well-designed data architecture should support business intelligence and analysis, automation, and AI—all of which can help organizations to quickly seize market opportunities, build customer value, drive major efficiencies, and respond to risks such as supply chain disruptions.

Strategy 85

Strategy Data Architecture Cost-Benefit Reporting 85

Cloudera

MAY 2, 2022

But increasingly at Cloudera, our clients are looking for a hybrid cloud architecture in order to manage compliance requirements. This is not just to implement specific governance rules — such as tagging, metadata management, access controls, or anonymization — but to prepare for the potential for rules to change in the future. .

Data Architecture 103

Data Architecture Metadata Digital Transformation Finance 103

IBM Big Data Hub

APRIL 27, 2022

Data fabric and data mesh are emerging data management concepts that are meant to address the organizational change and complexities of understanding, governing and working with enterprise data in a hybrid multicloud ecosystem. The good news is that both data architecture concepts are complimentary.

Management 79

Management Metadata Data Architecture Data Lake 79

Ontotext

APRIL 10, 2024

In most enterprises data teams lack a data map and data asset inventory and are often unaware of data that exists across the organization, its associated profile, quality and associated metadata. Teams can’t access data to build their business use cases. For example, a product data tag is basic metadata.

Metadata 52

Metadata Data Lake Data Warehouse Data Quality 52

Cloudera

MARCH 23, 2022

In fact, we recently announced the integration with our cloud ecosystem bringing the benefits of Iceberg to enterprises as they make their journey to the public cloud, and as they adopt more converged architectures like the Lakehouse. 1: Multi-function analytics . 3: Open Performance.

Metadata 91

Metadata Data Architecture Machine Learning Cost-Benefit 91

IBM Big Data Hub

SEPTEMBER 12, 2022

The journey starts with having a multimodal data governance framework that is underpinned by a robust data architecture like data fabric. To understand how a data fabric helps maintain compliance to privacy regulations, it’s helpful to look at some essential elements of that single pane of glass.

Metadata 91

Metadata Data Governance Enterprise Data Architecture 91

Cloudera

JUNE 7, 2022

Companies can now capitalize on the value in all their data, by delivering a hybrid data platform for modern data architectures with data anywhere. Cloudera Data Platform (CDP) is designed to address the critical requirements for modern data architectures today and tomorrow.

IT 112

IT Data Architecture Unstructured Data Big Data 112

Data Virtualization

DECEMBER 21, 2023

Reading Time: 2 minutes Data mesh is a modern, distributed data architecture in which different domain based data products are owned by different groups within an organization. And data fabric is a self-service data layer that is supported in an orchestrated fashion to serve.

Data Governance 59

Data Governance Data Architecture Data Integration Management 59

erwin

JANUARY 9, 2020

That’s because it’s the best way to visualize metadata , and metadata is now the heart of enterprise data management and data governance/ intelligence efforts. So here’s why data modeling is so critical to data governance. erwin Data Modeler: Where the Magic Happens.

Data Governance 144

Data Governance Modeling Metadata Unstructured Data 144

AWS Big Data

SEPTEMBER 7, 2023

Customers want to know how their data is being accessed, when it is being accessed, and who is accessing it. With exponential growth in data volume, centralized monitoring becomes challenging. It is also crucial to audit granular data access for security and compliance needs.

Metadata 106

Metadata Dashboards Metrics Visualization 106

Cloudera

JUNE 30, 2022

With Cloudera’s vision of hybrid data , enterprises adopting an open data lakehouse can easily get application interoperability and portability to and from on premises environments and any public cloud without worrying about data scaling. Why integrate Apache Iceberg with Cloudera Data Platform?

Data Lake 88

Data Lake Data Warehouse Data Architecture Metadata 88

Alation

JANUARY 9, 2023

The Australian Prudential Regulation Authority (APRA) released nonbinding standards covering data risk management. Another agency later also published a legally binding standard to strengthen risk management for financial institutions with specific language related to data architecture and IT infrastructure.

Risk Management 52

Risk Management Risk Management Metadata 52

Alation

JANUARY 25, 2022

Data fabric is now on the minds of most data management leaders. In our previous blog, Data Mesh vs. Data Fabric: A Love Story , we defined data fabric and outlined its uses and motivations. The data catalog is a foundational layer of the data fabric. The Power of Social Metadata.

Metadata 52

Metadata IT Metrics Data-driven 52

AWS Big Data

APRIL 26, 2023

This is a guest blog post co-written with Corey Johnson from Huron. Having an accurate and up-to-date inventory of all technical assets helps an organization ensure it can keep track of all its resources with metadata information such as their assigned oners, last updated date, used by whom, how frequently and more.

Metadata 52

Metadata Dashboards Visualization Consulting 52

erwin

OCTOBER 16, 2020

There’s a common denominator in what they’re all missing, and that is data intelligence. The erwin EDGE enables organizations to see how data flows through and impacts all their business, technology and data architectures. Parts of this blog are excerpted from my keynote on day No.

Internet of Things 145

Internet of Things Data-driven Uncertainty Data Governance 145

IBM Big Data Hub

AUGUST 24, 2022

Remote runtime data integration as-a-service execution capabilities for on-premises and multi-cloud execution. Multi-directional data movement topology with high volume and low-latency integration. Support for data governance. Metadata exchange with third party metadata management and governance tools.

Data Integration 79

Data Integration Metadata Data-driven Data Architecture 79

AWS Big Data

JULY 18, 2024

The current method is largely manual, relying on emails and general communication, which not only increases overhead but also varies from one use case to another in terms of data governance. Data domain producers publish data assets using datasource run to Amazon DataZone in the Central Governance account.

Data Lake 116

Data Lake Metadata Publishing Data-driven 116

erwin

MAY 13, 2020

The role of data modeling (DM) has expanded to support enterprise data management, including data governance and intelligence efforts. Metadata management is the key to managing and governing your data and drawing intelligence from it. Types of Data Models: Conceptual, Logical and Physical.

Data Governance 93

Data Governance Enterprise Modeling Management 93

Cloudera

MARCH 17, 2023

As the use of ChatGPT becomes more prevalent, I frequently encounter customers and data users citing ChatGPT’s responses in their discussions. I love the enthusiasm surrounding ChatGPT and the eagerness to learn about modern data architectures such as data lakehouses, data meshes, and data fabrics.

Unstructured Data 64

Unstructured Data Data Lake Data Warehouse Machine Learning 64

Data Virtualization

JANUARY 17, 2023

Reading Time: 2 minutes In recent years, there has been a growing interest in data architecture. One of the key considerations is how best to handle data, and this is where data mesh and data fabric come into play. But what are the key.

Data Architecture 52

Data Architecture Data Integration Management Metadata 52

Alation

MARCH 31, 2023

Ehtisham Zaidi, Gartner’s VP of data management, and Robert Thanaraj, Gartner’s director of data management, gave an update on the fabric versus mesh debate in light of what they call the “active metadata era” we’re currently in. The foundations of successful data governance The state of data governance was also top of mind.

Data Analytics 52

Data Analytics Analytics Metadata Data Governance 52

erwin

JULY 12, 2019

Modern, strategic data governance , which involves both IT and the business, enables organizations to plan and document how they will discover and understand their data within context, track its physical existence and lineage, and maximize its security, quality and value. Five Steps to GDPR/CCPA Compliance. How erwin Can Help.

Data Governance 110

Data Governance Management Metadata Risk Management 110