AWS Big Data

OCTOBER 4, 2023

Due to this limitation, the cost of failures of long-running extract, transform, and load (ETL) and batch queries on Trino was high in terms of completion time, compute wastage, and spend. These new enhancements in Trino with Amazon EMR provide improved resiliency for running ETL and batch workloads on Spot Instances with reduced costs.

Big Data 114

Big Data Optimization Data-driven Management 114

erwin

NOVEMBER 6, 2020

Import existing Excel or CSV files, use the drag-and-drop feature to extract the mappings from your ETL scripts, or manually populate the inventory to then be visualized with the lineage analyzer. Data Cataloging: Catalog and sync metadata with data management and governance artifacts according to business requirements in real time.

Data Governance 97

Data Governance Metadata Data-driven Visualization 97

AWS Big Data

JULY 26, 2023

The following are common asks from our customers: Is it possible to develop and test AWS Glue data integration jobs on my local laptop? The software development lifecycle on AWS defines the following six phases: Plan, Design, Implement, Test, Deploy, and Maintain. Test In the testing phase, you check the implementation for bugs.

Data Integration 97

Data Integration Snapshot Testing Visualization 97

AWS Big Data

NOVEMBER 15, 2023

BMW Cloud Efficiency Analytics (CLEA) is a homegrown tool developed within the BMW FinOps CoE (Center of Excellence) aiming to optimize and reduce costs across all these accounts. In this post, we explore how the BMW Group FinOps CoE implemented their Cloud Efficiency Analytics tool (CLEA), powered by Amazon QuickSight and Amazon Athena.

Analytics 111

Analytics Dashboards Metadata Data Warehouse 111

AWS Big Data

JULY 18, 2024

A data domain producer maintains its own ETL stack using AWS Glue , AWS Lambda to process, AWS Glue Databrew to profile the data and prepare the data asset (data product) before cataloguing it into AWS Glue Data Catalog in their account. Producers ingest data into their S3 buckets through pipelines they manage, own, and operate.

Data Lake 130

Data Lake Publishing Metadata Data-driven 130

AWS Big Data

JULY 25, 2024

Amazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that makes it straightforward and cost-effective to analyze all your data using standard SQL and your existing extract, transform, and load (ETL); business intelligence (BI); and reporting tools. Add any necessary tags to the instance.

Management 116

Management Data Warehouse Big Data Testing 116

AWS Big Data

JUNE 6, 2023

In order to mature our data marts, it became clear that we needed to provide Analysts and other data consumers with all tracked digital analytics data in our DWH as they depend on it for analyses, reporting, campaign evaluation, product development and A/B testing. We made sure to add appropriate tags for cost monitoring.

Analytics 59

Analytics Data Lake Testing Optimization 59

Alation

OCTOBER 27, 2022

This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. What tools do you use? For this use case, we leveraged Fivetran, dbt, and Alation to build an internal tool that estimates unique buyer personas. How does this help the end user?

Metrics 52

Metrics Dashboards Sales Reporting 52

Alation

JANUARY 25, 2022

But social extends even further; it includes the comments users make, assets they flag as favorites, the watches they set on assets, who they tag and how, and their level of expertise. At its best, a data catalog should empower data analysts, scientists, and anyone curious about data with tools to explore and understand it. Simply put?

Metadata 52

Metadata IT Data-driven Metrics 52

AWS Big Data

MAY 9, 2025

SageMaker Lakehouse organizes data using logical containers called catalogs , enabling teams to seamlessly query and analyze data across their entire ecosystemfrom S3 data lakes to Amazon Redshift warehousesusing familiar Apache Iceberg compatible tools. Under LF-Tags or catalog resources , select Named Data Catalog resources.

Data Lake 61

Data Lake Data Warehouse Marketing Management 61

AWS Big Data

MARCH 21, 2025

Traditionally, answering this question would involve multiple data exports, complex extract, transform, and load (ETL) processes, and careful data synchronization across systems. Users can write data to managed RMS tables using Iceberg APIs, Amazon Redshift, or Zero-ETL ingestion from supported data sources.

Data Warehouse 111

Data Warehouse Metadata Publishing Sales 111