AWS Big Data

NOVEMBER 11, 2024

This post focuses on introducing an active-passive approach using a snapshot and restore strategy. Snapshot and restore in OpenSearch Service The snapshot and restore strategy in OpenSearch Service involves creating point-in-time backups, known as snapshots , of your OpenSearch domain.

Snapshot 91

Snapshot Strategy Dashboards Data Lake 91

AWS Big Data

OCTOBER 11, 2024

Snapshots are crucial for data backup and disaster recovery in Amazon OpenSearch Service. These snapshots allow you to generate backups of your domain indexes and cluster state at specific moments and save them in a reliable storage location such as Amazon Simple Storage Service (Amazon S3). Snapshots are not instantaneous.

Snapshot 85

Snapshot Dashboards Management Testing 85

AWS Big Data

OCTOBER 18, 2023

in Amazon OpenSearch Service , we introduced Snapshot Management , which automates the process of taking snapshots of your domain. Snapshot Management helps you create point-in-time backups of your domain using OpenSearch Dashboards, including both data and configuration settings (for visualizations and dashboards).

Snapshot 88

Snapshot Management Dashboards Data Processing 88

AWS Big Data

NOVEMBER 21, 2024

The AWS Glue Data Catalog supports automatic table optimization of Apache Iceberg tables, including compaction , snapshots, and orphan data management. The data compaction optimizer constantly monitors table partitions and kicks off the compaction process when the threshold is exceeded for the number of files and file sizes.

Optimization 79

Optimization Snapshot Metadata Software 79

AWS Big Data

APRIL 3, 2024

The AWS Glue crawler generates and updates Iceberg table metadata and stores it in AWS Glue Data Catalog for existing Iceberg tables on an S3 data lake. Snowflake integrates with AWS Glue Data Catalog to retrieve the snapshot location. Snowflake can query across Iceberg and Snowflake table formats.

Data Lake 100

Data Lake Snapshot Metadata Data Architecture 100

AWS Big Data

JUNE 28, 2023

Create a role in the target account with the following permissions: { "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "redshift:DescribeClusters", "redshift-serverless:ListNamespaces" ], "Resource":[ "*" ] } ] } The role must have the following trust policy, which specifies the target account ID. Choose Create policy.

Data Warehouse 89

Data Warehouse Analytics Metrics Dashboards 89

AWS Big Data

APRIL 10, 2024

Create a role in the target account with the following permissions: { "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action":[ "redshift:DescribeClusters", "redshift-serverless:ListNamespaces" ], "Resource":[ "*" ] } ] } The role must have the following trust policy, which specifies the target account ID.

Data Warehouse 100

Data Warehouse Analytics Metrics Snapshot 100