Cloudera

SEPTEMBER 29, 2022

Each of these trends claim to be complete models for their data architectures to solve the “everything everywhere all at once” problem. Data teams are confused as to whether they should get on the bandwagon of just one of these trends or pick a combination. First, we describe how data mesh and data fabric could be related.

Data Architecture 74

Data Architecture Data Warehouse Metadata Sales 74

AWS Big Data

FEBRUARY 26, 2025

The Gartner Magic Quadrant evaluates 20 data integration tool vendors based on two axesAbility to Execute and Completeness of Vision. Discover, prepare, and integrate all your data at any scale AWS Glue is a fully managed, serverless data integration service that simplifies data preparation and transformation across diverse data sources.

Data Integration 81

Data Integration Data Lake Data Warehouse Unstructured Data 81

DataKitchen

DECEMBER 20, 2020

They give data scientists tools to instantiate development sandboxes on demand. They automate the data operations pipeline and create platforms used to test and monitor data from ingestion to published charts and graphs.

Data-driven 147

Data-driven Manufacturing Data Architecture Data Analytics 147

AWS Big Data

DECEMBER 4, 2024

With this launch, you can query data regardless of where it is stored with support for a wide range of use cases, including analytics, ad-hoc querying, data science, machine learning, and generative AI. We’ve simplified data architectures, saving you time and costs on unnecessary data movement, data duplication, and custom solutions.

Data Analytics 85

Data Analytics Analytics Data Lake Data Quality 85

AWS Big Data

JULY 18, 2024

This blog post introduces Amazon DataZone and explores how VW used it to build their data mesh to enable streamlined data access across multiple data lakes. Amazon DataZone projects enable collaboration with teams through data assets and the ability to manage and monitor data assets across projects.

Data Lake 122

Data Lake Publishing Metadata Data-driven 122

Cloudera

JUNE 7, 2022

Companies can now capitalize on the value in all their data, by delivering a hybrid data platform for modern data architectures with data anywhere. Cloudera Data Platform (CDP) is designed to address the critical requirements for modern data architectures today and tomorrow.

IT 112

IT Data Architecture Unstructured Data Big Data 112

Cloudera

FEBRUARY 22, 2022

Over the past decade, the successful deployment of large scale data platforms at our customers has acted as a big data flywheel driving demand to bring in even more data, apply more sophisticated analytics, and on-board many new data practitioners from business analysts to data scientists.

Snapshot 110

Snapshot Metadata Cost-Benefit Data Architecture 110

erwin

JULY 12, 2019

Modern, strategic data governance , which involves both IT and the business, enables organizations to plan and document how they will discover and understand their data within context, track its physical existence and lineage, and maximize its security, quality and value.

Data Governance 110

Data Governance Management Metadata Risk Management 110

AWS Big Data

DECEMBER 4, 2024

While traditional extract, transform, and load (ETL) processes have long been a staple of data integration due to its flexibility, for common use cases such as replication and ingestion, they often prove time-consuming, complex, and less adaptable to the fast-changing demands of modern data architectures.

Data Integration 97

Data Integration Data Lake Statistics Data-driven 97

CIO Business Intelligence

NOVEMBER 29, 2022

A sea of complexity For years, data ecosystems have gotten more complex due to discrete (and not necessarily strategic) data-platform decisions aimed at addressing new projects, use cases, or initiatives. Layering technology on the overall data architecture introduces more complexity.

Data Architecture 111

Data Architecture Data Integration IoT Data-driven 111

AWS Big Data

DECEMBER 19, 2024

The prod-hema-data-catalog is the production-grade catalog that supports data sharing across production services and, in some cases, pre-production services. The following diagram illustrates the architecture of both accounts. Oghosa Omorisiagbon is a Senior Data Engineer at HEMA.

Data Governance 52

Data Governance Publishing Data-driven Metadata 52

CIO Business Intelligence

JUNE 23, 2022

Companies can now capitalize on the value in all their data, by delivering a hybrid data platform for modern data architectures with data anywhere. Cloudera Data Platform (CDP) is designed to address the critical requirements for modern data architectures today and tomorrow.

IT 97

IT Data Architecture Unstructured Data Big Data 97

DataKitchen

OCTOBER 11, 2021

First, you must understand the existing challenges of the data team, including the data architecture and end-to-end toolchain. Figure 2: Example data pipeline with DataOps automation. In this project, I automated data extraction from SFTP, the public websites, and the email attachments.

Testing 152

Testing Metadata Dashboards Statistics 152

AWS Big Data

JULY 25, 2024

Instead of a central data platform team with a data warehouse or data lake serving as the clearinghouse of all data across the company, a data mesh architecture encourages distributed ownership of data by data producers who publish and curate their data as products, which can then be discovered, requested, and used by data consumers.

Data Lake 106

Data Lake Metadata Sales Publishing 106

IBM Big Data Hub

APRIL 27, 2022

Data fabric and data mesh are emerging data management concepts that are meant to address the organizational change and complexities of understanding, governing and working with enterprise data in a hybrid multicloud ecosystem. The good news is that both data architecture concepts are complimentary.

Management 79

Management Metadata Data Architecture Data Lake 79

Cloudera

OCTOBER 7, 2021

Those decentralization efforts appeared under different monikers through time, e.g., data marts versus data warehousing implementations (a popular architectural debate in the era of structured data) then enterprise-wide data lakes versus smaller, typically BU-Specific, “data ponds”.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

CIO Business Intelligence

NOVEMBER 1, 2023

It’s published two new resources for using BTP — a guidance framework with methodologies and reference architectures, and a developers’ guide including building blocks and step-by-step guides — and released an open-source SDK for building extensions on BTP.

Enterprise 132

Enterprise Cost-Benefit Unstructured Data Software 132

AWS Big Data

JANUARY 31, 2023

In this post, we are excited to summarize the features that the AWS Glue Data Catalog, AWS Glue crawler, and Lake Formation teams delivered in 2022. Whether you are a data platform builder, data engineer, data scientist, or any technology leader interested in data lake solutions, this post is for you.

Data Lake 89

Data Lake Data Governance Data Architecture Machine Learning 89

Cloudera

SEPTEMBER 8, 2022

With the volumes of data in telco accelerating with the rapid advancement of 5G and IoT, the time is now to modernize the data architecture. . Cloudera has been working closely with the TM Forum for many years now, and has in particular been playing a leadership role in Data Governance and AI Governance collaboration workgroups.

Data Governance 79

Data Governance B2B Data Architecture IoT 79

AWS Big Data

JANUARY 30, 2025

We configured our Amazon AppFlow flows to store the output data in Amazon S3, then used an EventBridge rule based on an End Flow Run Report event (which is an event which is published when a flow run is complete) to trigger a load into Amazon Redshift using a COPY statement. The following Diagram 4 shows this workflow.

Data Warehouse 52

Data Warehouse Data Architecture Machine Learning Data Transformation 52

Cloudera

JULY 29, 2024

Enterprise stream management is the ability to manage an intermediary that can broker real-time data between any number of “publishing” sources and “subscribing” destinations. This capability is the backbone of building real-time use cases, and it eliminates the need to build sprawling point-to-point connections across the enterprise.

Enterprise 69

Enterprise Data Architecture Manufacturing Key Performance Indicator 69

Smarten

JUNE 1, 2022

The integrated solution provides access to data sources and data warehouses using a robust data architecture with single-tenant or multi-tenant modes and flexible deployment via public or private cloud, or via on-premises hardware, so the business can deploy anywhere with no environmental dependencies.

Data Warehouse 78

Data Warehouse Software Data Architecture Dashboards 78

Cloudera

DECEMBER 21, 2021

We are excited to offer in Tech Preview this born-in-the-cloud table format that will help future proof data architectures at many of our public cloud customers. Modernizing pipelines. And we look forward to contributing even more CDP operators to the community in the coming months. Performance boost with Spark 3.1.

Snapshot 118

Snapshot Data-driven Optimization Management 118

AWS Big Data

JUNE 10, 2024

The data resides on Amazon S3, which reduces the storage costs significantly. Centralized catalog for published data – Multiple producers release data currently governed by their respective entities. For consumer access, a centralized catalog is necessary where producers can publish their data assets.

Data Lake 96

Data Lake Metadata Data Warehouse Data Processing 96

Cloudera

NOVEMBER 18, 2020

We also celebrated the first-ever winner of the Data Impact Achievement Award — a new award category that recognizes one customer who has consistently achieved transformation across their business, pursuing a diverse set of use cases and creating a culture of data-driven innovation. . Data Impact Achievement Award.

Internet Publishing and Broadcasting 123

Internet Publishing and Broadcasting Data-driven Broadcasting Digital Transformation 123

IBM Big Data Hub

JULY 5, 2022

In today’s world of complex data architectures and emerging technologies, databases can sometimes be undervalued and unrecognized. Back in the 1960s and 70s, vast amounts of data were stored in the world’s new mainframe computers—many of them IBM System/360 machines—and had become a problem. They were expensive.

Data Lake 86

Data Lake Data Warehouse Publishing Structured Data 86

CIO Business Intelligence

MARCH 19, 2025

Integrating ESG into data decision-making CDOs should embed sustainability into data architecture, ensuring that systems are designed to optimize energy efficiency, minimize unnecessary data replication and promote ethical data use.

IT 59

IT Data Governance Data-driven Metrics 59

AWS Big Data

APRIL 17, 2024

Now the AaaS provider has multiple methods to deliver insights to their customers: Option 1 – The enriched data with insights is shared directly with the customer’s Redshift instance using the Amazon Redshift data sharing feature. End-users consume data using business intelligence (BI) tools and analytics applications.

Data Warehouse 113

Data Warehouse Analytics Cost-Benefit Data Processing 113

TDAN

JUNE 21, 2022

Twenty-five years ago today, I published the first issue of The Data Administration Newsletter. It only took a few months to recognize that there was an audience for an “online” publication focused on data administration. […].

Publishing 78

Publishing Data Quality Data Architecture IT 78

AWS Big Data

SEPTEMBER 21, 2023

Integrating Satori with Amazon Redshift accelerates organizations’ ability to make use of their data to generate business value. This faster time-to-value is achieved by enabling companies to manage data access more efficiently and effectively. Lisa Levy is a Content Specialist at Satori.

Data Warehouse 98

Data Warehouse Interactive Data Architecture Data-driven 98

AWS Big Data

AUGUST 1, 2024

Success criteria alignment by all stakeholders (producers, consumers, operators, auditors) is key for successful transition to a new Amazon Redshift modern data architecture. The success criteria are the key performance indicators (KPIs) for each component of the data workflow.

Data Warehouse 99

Data Warehouse KPI Optimization Cost-Benefit 99

AWS Big Data

MARCH 10, 2023

Tracking data changes and rollback Build your transactional data lake on AWS You can build your modern data architecture with a scalable data lake that integrates seamlessly with an Amazon Redshift powered cloud warehouse. He is passionate about data and emerging technologies in analytics.

Data Lake 116

Data Lake Sales Data Warehouse Snapshot 116

Cloudera

JANUARY 30, 2024

Cloudera provides end-to-end data life cycle management on a hybrid data platform, which includes all the building blocks needed to build a data strategy for trusted data in manufacturing. The post Achieving Trusted AI in Manufacturing appeared first on Cloudera Blog.

Manufacturing 81

Manufacturing Contextual Data IoT Internet of Things 81

Sisense

SEPTEMBER 3, 2020

This includes database modeling, metrics definition, dashboard design , and creating and publishing executive reports. ROI (return on investment) is also a key concern, as business analysts apply their data-related activities to finance, marketing, and risk management, for instance. See an example: Explore Dashboard.

Statistics 97

Statistics Metrics Visualization Finance 97

Data Virtualization

DECEMBER 8, 2022

Reading Time: 3 minutes While cleaning up our archive recently, I found an old article published in 1976 about data dictionary/directory systems (DD/DS). Nowadays, we no longer use the term DD/DS, but “data catalog” or simply “metadata system”. It was written by L.

Metadata 69

Metadata IT Data Integration Publishing 69

AWS Big Data

SEPTEMBER 7, 2023

Amazon QuickSight enables organizations to build visualizations, perform case-by-case analysis, and quickly get business insights from their data anytime, on any device. You can use other business intelligence (BI) tools that integrate with Athena to build dashboards and share or publish them to provide timely insights.

Metadata 111

Metadata Dashboards Metrics Visualization 111

Peter James Thomas

JANUARY 8, 2020

White Papers can be based on themes arising from articles published here, they can feature findings from de novo research commissioned in the data arena, or they can be on a topic specifically requested by the client. Sometimes the labels of these are white [1] as well as the paper.

Consulting 61

Consulting Data-driven Publishing Data Strategy 61