Data Lake Experimentation Snapshot 
AWS Big Data
DECEMBER 12, 2024
In the context of comprehensive data governance, Amazon DataZone offers organization-wide data lineage visualization using Amazon Web Services (AWS) services, while dbt provides project-level lineage through model analysis and supports cross-project integration between data lakes and warehouses.
AWS Big Data
OCTOBER 19, 2023
Data lakes and data warehouses are two of the most important data storage and management technologies in a modern data architecture. Data lakes store all of an organization’s data, regardless of its format or structure.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
AWS Big Data
MAY 24, 2023
When you build your transactional data lake using Apache Iceberg to solve your functional use cases, you need to focus on operational use cases for your S3 data lake to optimize the production environment. You can use either the AWS Glue Data Catalog (recommended) or a Hive catalog for Iceberg tables.
O'Reilly on Data
OCTOBER 19, 2021
It has far-reaching implications as to how such applications should be developed and by whom: ML applications are directly exposed to the constantly changing real world through data, whereas traditional software operates in a simplified, static, abstract world which is directly constructed by the developer. This approach is not novel.
AWS Big Data
JULY 20, 2023
With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.
AWS Big Data
JULY 3, 2023
Terminology Let’s first discuss some of the terminology used in this post: Research data lake on Amazon S3 – A data lake is a large, centralized repository that allows you to manage all your structured and unstructured data at any scale. This is where the tagging feature in Apache Iceberg comes in handy.
AWS Big Data
JANUARY 24, 2023
The utility for cloning and experimentation is available in the open-sourced GitHub repository. This solution only replicates metadata in the Data Catalog, not the actual underlying data. This ensures that the data lake will still be functional in another Region if Lake Formation has an availability issue.
Expert insights. Personalized for you.
97 
Let's personalize your content