AWS Big Data

JUNE 10, 2024

In this post, we delve into the key aspects of using Amazon EMR for modern data management, covering topics such as data governance, data mesh deployment, and streamlined data discovery. Organizations have multiple Hive data warehouses across EMR clusters, where the metadata gets generated.

Data Lake 113

Data Lake Metadata Data Warehouse Data Processing 113

AWS Big Data

APRIL 28, 2023

The third post will show how end-users can consume data from their tool of choice, without compromising data governance. When building a scalable data architecture on AWS, giving autonomy and ownership to the data domains are crucial for the success of the platform.

Data Governance 72

Data Governance Management Data-driven Analytics 72

Cloudera

NOVEMBER 18, 2020

The technological linchpin of its digital transformation has been its Enterprise Data Architecture & Governance platform. It hosts over 150 big data analytics sandboxes across the region with over 200 users utilizing the sandbox for data discovery.

Internet Publishing and Broadcasting 124

Internet Publishing and Broadcasting Data-driven Broadcasting Digital Transformation 124

Cloudera

OCTOBER 19, 2023

While navigating so many simultaneous data-dependent transformations, they must balance the need to level up their data management practices—accelerating the rate at which they ingest, manage, prepare, and analyze data—with that of governing this data.

Data-driven 81

Data-driven Data Architecture Predictive Modeling Digital Transformation 81

TDAN

MAY 31, 2022

In fact, according to Gartner, “60 percent of the data used for the development of AI and analytics projects will be synthetically generated.”[1] 1] I had never heard about synthetic data until I listened to the AI Today podcast, hosted by Kathleen Welch […].

Data Processing 105

Data Processing Analytics Data Architecture Data Governance 105

CIO Business Intelligence

FEBRUARY 28, 2024

And not only do companies have to get all the basics in place to build for analytics and MLOps, but they also need to build new data structures and pipelines specifically for gen AI. And for some use cases, an expensive, high-end commercial LLM might not be required since a locally-hosted open source model might suffice.

IT 131

IT Consulting Modeling Enterprise 131

AWS Big Data

JANUARY 22, 2024

Discussions with users showed they were happier to have faster access to data in a simpler way, a more structured data organization, and a clear mapping of who the producer is. A lot of progress has been made to advance their data-driven culture (data literacy, data sharing, and collaboration across business units).

Data-driven 115

Data-driven Advertising Metadata Data Architecture 115

AWS Big Data

MARCH 10, 2023

Tracking data changes and rollback Build your transactional data lake on AWS You can build your modern data architecture with a scalable data lake that integrates seamlessly with an Amazon Redshift powered cloud warehouse. Data can be organized into three different zones, as shown in the following figure.

Data Lake 122

Data Lake Sales Data Warehouse Snapshot 122

AWS Big Data

MARCH 27, 2023

Data producers can use the data mesh platform to create datasets and share them across business teams to ensure data availability, reliability, and interoperability across functions and data subject areas. The data mesh producer account hosts the encrypted S3 bucket, which is shared with the central governance account.

Data Lake 98

Data Lake Data-driven Management Data Architecture 98

AWS Big Data

MARCH 26, 2024

In this post, we discuss how you can use purpose-built AWS services to create an end-to-end data strategy for C360 to unify and govern customer data that address these challenges. Strategize based on how your teams explore data, run analyses, wrangle data for downstream requirements, and visualize data at different levels.

Data Strategy 94

Data Strategy Strategy Data Warehouse Prescriptive Analytics 94

erwin

SEPTEMBER 16, 2024

The gold standard in data modeling solutions for more than 30 years continues to evolve with its latest release, highlighted by: PostgreSQL 16.x The gold standard in data modeling solutions for more than 30 years continues to evolve with its latest release, highlighted by: PostgreSQL 16.x

Modeling 52

Modeling Visualization Data Governance Data Architecture 52

Alation

FEBRUARY 20, 2020

Experts who understand certain datasets often play the stewardship role of ensuring that data consumers can make accurate and effective use of data. More recently, data governance initiatives have started to assign formal stewardship responsibility. In the release of Alation 4.0, In the release of Alation 4.0,

Metadata 52

Metadata Enterprise Data Processing Data Architecture 52

Andrew White

JANUARY 9, 2022

On Thursday January 6th I hosted Gartner’s 2022 Leadership Vision for Data and Analytics webinar. Most of D&A concerns and activities are done within EA in the Info/Data architecture domain/phases. Could you precise to which complementary research you mentioned when you talked about a data governance survey ?

Analytics 64

Analytics Measurement Data-driven Modeling 64

CIO Business Intelligence

DECEMBER 11, 2024

The proposed model illustrates the data management practice through five functional pillars: Data platform; data engineering; analytics and reporting; data science and AI; and data governance. Thats free money given to cloud providers and creates significant issues in end-to-end value generation.

Management 111

Management Data Governance Data Science Reporting 111

AWS Big Data

FEBRUARY 27, 2024

Four-layered data lake and data warehouse architecture – The architecture comprises four layers, including the analytical layer, which houses purpose-built facts and dimension datasets that are hosted in Amazon Redshift. It played a critical role in enforcing data access controls and implementing data policies.

Data-driven 111

Data-driven Data Lake Data Quality Data Governance 111

AWS Big Data

DECEMBER 19, 2024

HEMA has a bespoke enterprise architecture, built around the concept of services. Each service is hosted in a dedicated AWS account and is built and maintained by a product owner and a development team, as illustrated in the following figure. Implementing robust data governance is challenging.

Data Governance 52

Data Governance Publishing Data-driven Metadata 52

AWS Big Data

JULY 14, 2023

In this post, we discuss how the Amazon Finance Automation team used AWS Lake Formation and the AWS Glue Data Catalog to build a data mesh architecture that simplified data governance at scale and provided seamless data access for analytics, AI, and machine learning (ML) use cases.

Finance 98

Finance Metadata Big Data Recreation/Entertainment 98

Cloudera

OCTOBER 7, 2021

Those decentralization efforts appeared under different monikers through time, e.g., data marts versus data warehousing implementations (a popular architectural debate in the era of structured data) then enterprise-wide data lakes versus smaller, typically BU-Specific, “data ponds”.

Metadata 129

Metadata Cost-Benefit Enterprise Interactive 129

Cloudera

FEBRUARY 22, 2021

The Cloudera Data Platform (CDP) represents a paradigm shift in modern data architecture by addressing all existing and future analytical needs. Cloudera Data Catalog (part of SDX) replaces data governance tools to facilitate centralized data governance (data cataloging, data searching / lineage, tracking of data issues etc. ).

Cost-Benefit 84

Cost-Benefit Data Warehouse Data Processing Data Governance 84

erwin

APRIL 16, 2024

The data mesh framework In the dynamic landscape of data management, the search for agility, scalability, and efficiency has led organizations to explore new, innovative approaches. One such innovation gaining traction is the data mesh framework. This empowers individual teams to own and manage their data.

Metadata 52

Metadata Data Quality Data Governance Modeling 52

AWS Big Data

DECEMBER 21, 2023

Overview of solution As a data-driven company, smava relies on the AWS Cloud to power their analytics use cases. smava ingests data from various external and internal data sources into a landing stage on the data lake based on Amazon Simple Storage Service (Amazon S3).

Data Lake 115

Data Lake Data Warehouse B2B Data-driven 115

CIO Business Intelligence

JANUARY 20, 2023

A big part of preparing data to be shared is an exercise in data normalization, says Juan Orlandini, chief architect and distinguished engineer at Insight Enterprises. Data formats and data architectures are often inconsistent, and data might even be incomplete.

Data Lake 135

Data Lake Data-driven Finance Data Architecture 135