Remove 2023 Remove Blog Remove Snapshot
article thumbnail

Chart Snapshot: Barcode Plot

The Data Visualisation Catalogue

— VizWiz ‘Avengers’ characters’ appearances over time How the ‘Avengers’ Line-up Has Changed Over the Years — Wall Street Journal Multiple Income Households Flowingdata / Nathan Yau The Corruption Perceptions Index 2023 Week 35 | Power BI: Create a Faceted Instance Chart — Workout Wednesday / Meagan Longoria The post Chart (..)

article thumbnail

Amazon OpenSearch Service H1 2023 in review

AWS Big Data

Since its release in January 2021, the OpenSearch project has released 14 versions through June 2023. In this post, we provide a review of all the exciting features releases in OpenSearch Service in the first half of 2023. In July 2023, we previewed support for a third collection type: vector search.

Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

Improve operational efficiencies of Apache Iceberg tables built on Amazon S3 data lakes

AWS Big Data

Update your-iceberg-storage-blog in the following configuration with the bucket that you created to test this example. S3FileIO", "spark.sql.catalog.dev.warehouse":"s3://<your-iceberg-storage-blog>/iceberg/", "spark.sql.catalog.dev.s3.write.tags.write-tag-name":"created", write.tags.write-tag-name and s3.delete.tags.delete-tag-name

article thumbnail

Migrate an existing data lake to a transactional data lake using Apache Iceberg

AWS Big Data

An in-place migration can be performed in either of two ways: Using add_files : This procedure adds existing data files to an existing Iceberg table with a new snapshot that includes the files. Unlike migrate or snapshot, add_files can import files from a specific partition or partitions and doesn’t create a new Iceberg table.

Data Lake 122
article thumbnail

Use Apache Iceberg in a data lake to support incremental data processing

AWS Big Data

Whenever there is an update to the Iceberg table, a new snapshot of the table is created, and the metadata pointer points to the current table metadata file. At the top of the hierarchy is the metadata file, which stores information about the table’s schema, partition information, and snapshots. Choose Advanced options.

Data Lake 137
article thumbnail

Smarten Augmented Analytics is Named as a Representative Vendor in Gartner® 2023 ‘Market Guide for Augmented Analytics, Published October, 2023!

Smarten

Smarten is pleased to announce that its Smarten Augmented Analytics solution is included as a Representative Vendor in the Market Guide for Augmented Analytics Published October 2, 2023 (ID G00780764). The Smarten Cloud Software-as-a-Service offering includes all of these features and is available for free evaluation.

article thumbnail

Apache Iceberg optimization: Solving the small files problem in Amazon EMR

AWS Big Data

To activate the automatic compaction process, add a new record to the existing Iceberg table using a Spark insert: spark.sql(""" Insert into dev.db.sensor_data_iceberg_format values(999123, 86, 'PASS', timestamp'2023-07-26 12:50:25') """) Navigate to the Amazon EMR console to check the cluster steps. impl":"org.apache.iceberg.aws.s3.S3FileIO",