CIO Business Intelligence

DECEMBER 11, 2024

This article proposes a methodology for organizations to implement a modern data management function that can be tailored to meet their unique needs. By modern, I refer to an engineering-driven methodology that fully capitalizes on automation and software engineering best practices.

Management 111

Management Data Governance Data Science Reporting 111

AWS Big Data

JANUARY 6, 2025

In this post, we show you how to establish the data ingestion pipeline between Google Analytics 4, Google Sheets, and an Amazon Redshift Serverless workgroup. He has helped customers build scalable data warehousing and big data solutions for over 16 years. In his spare time, he loves reading, walking, and doing yoga.

Analytics 98

Analytics Data Warehouse Big Data Metrics 98

Smart Data Collective

JUNE 16, 2025

He is passionate about covering topics like big data, business intelligence, startups & entrepreneurship. Artificial Intelligence for eCommerce: A Closer Look Artificial Intelligence How To Get An Award Winning Giveaway Bot Big Data Chatbots Exclusive Quick Link About Contact Privacy Follow US © 2008-25 SmartData Collective.

Business Intelligence 84

Business Intelligence Modeling Machine Learning Big Data 84

AWS Big Data

DECEMBER 16, 2024

About the Authors Chiho Sugimoto is a Cloud Support Engineer on the AWS Big Data Support team. She is passionate about helping customers build data lakes using ETL workloads. Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He strives to delight customers by helping his team build software.

Visualization 91

Visualization Data Processing Testing Publishing 91

AWS Big Data

NOVEMBER 14, 2024

The landscape of big data management has been transformed by the rising popularity of open table formats such as Apache Iceberg, Apache Hudi, and Linux Foundation Delta Lake. These formats, designed to address the limitations of traditional data storage systems, have become essential in modern data architectures.

Metadata 122

Metadata Data Warehouse Big Data Data Lake 122

AWS Big Data

NOVEMBER 21, 2024

Amazon EMR is a cloud big data platform for petabyte-scale data processing, interactive analysis, streaming, and machine learning (ML) using open source frameworks such as Apache Spark , Presto and Trino , and Apache Flink. About the Authors Garima Arora is a Software Development Engineer for Amazon EMR at Amazon Web Services.

Metrics 76

Metrics Machine Learning Strategy Big Data 76

AWS Big Data

NOVEMBER 27, 2024

Credits to: Vesa Pehkonen, Noah Staveley, Assane Diop, Naveen Tatikonda About the Authors Mulugeta Mammo is a Senior Software Engineer, and currently leads the OpenSearch Optimization team at Intel. Vamshi Vijay Nakkirtha is a software engineering manager working on the OpenSearch Project and Amazon OpenSearch Service.

Cost-Benefit 123

Cost-Benefit Machine Learning Optimization Software 123

AWS Big Data

DECEMBER 12, 2024

Has many years of experience in big data, enterprise digital transformation research and development, consulting, and project management across telecommunications, entertainment, and financial industries. He has more than 25 years of experience in IT industry, software development and solution architecture.

Snapshot 96

Snapshot Recreation/Entertainment Experimentation Data Lake 96

AWS Big Data

DECEMBER 4, 2024

About the Authors Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He is responsible for building software artifacts to help customers. Stuti Deshpande is a Big Data Specialist Solutions Architect at AWS. She has extensive experience in big data, ETL, and analytics.

Data Lake 76

Data Lake Cost-Benefit Data Integration Data Warehouse 76

AWS Big Data

NOVEMBER 7, 2024

Allow running third party software – Because BladeBridge Analyzer is a third-party software, MacOS may raise a warning or an error when running Analyzer. He has helped customers build scalable data warehousing and big data solutions for over 16 years. For Mac and Linux users, run chmod 755./bbanalyzer

Data Warehouse 106

Data Warehouse Reporting Big Data Data Lake 106

AWS Big Data

DECEMBER 4, 2024

SageMaker brings together widely adopted AWS ML and analytics capabilities—virtually all of the components you need for data exploration, preparation, and integration; petabyte-scale big data processing; fast SQL analytics; model development and training; governance; and generative AI development.

Data Analytics 103

Data Analytics Analytics Data Lake Data Quality 103

AWS Big Data

OCTOBER 30, 2024

Users can begin ingesting data to Redshift from Amazon S3 with simple SQL commands and gain access to the most up-to-date data without the need for third-party tools or custom implementation. He has worked with building data warehouses and big data solutions for over 15+ years.

Data Warehouse 109

Data Warehouse Sales Data Lake Recreation/Entertainment 109

Smart Data Collective

JULY 16, 2025

Reading: Turning Data Into Decisions: How Analytics Improves Transportation Strategy Share Notification Font Resizer Aa Font Resizer Aa Search About Help Privacy Follow US © 2008-23 SmartData Collective. Andrej Kovacevic 3 Min Read Licensed Photo from Microsoft Stock Images SHARE Transportation networks generate a constant stream of data.

Strategy 68

Strategy Analytics Big Data IoT 68

AWS Big Data

OCTOBER 21, 2024

For producers seeking collaboration with partners, AWS Clean Rooms facilitates secure collaboration and analysis of collective datasets without the need to share or duplicate underlying data. He has a background in software development and hybrid architectures, and is passionate about helping customers modernize their cloud architecture.

Sales 116

Sales Data-driven Data Processing Key Performance Indicator 116

AWS Big Data

JUNE 18, 2025

We encourage you to evaluate RocksDB for your use cases, particularly if you’re experiencing memory pressure issues with the default state store or need to handle large amounts of state data in your streaming applications. About the authors Melody Yang is a Senior Big Data Solution Architect for Amazon EMR at AWS.

Optimization 110

Optimization Snapshot Metrics Big Data 110

AWS Big Data

DECEMBER 27, 2024

To stay informed, subscribe to the AWS Big Data Blogs RSS feed , where you can find updates on the EMR runtime for Spark and Iceberg, as well as tips on configuration best practices and tuning recommendations. About the Authors Atul Felix Payapilly is a software development engineer for Amazon EMR at Amazon Web Services.

Cost-Benefit 106

Cost-Benefit Testing Metrics Optimization 106

AWS Big Data

NOVEMBER 7, 2024

He has a strong and proven technical background in software development and architecture, balanced with a drive to deliver commercially successful products. Erol Murtezaoglu , a Technical Product Manager at AWS, is an inquisitive and enthusiastic thinker with a drive for self-improvement and learning.

Metadata 106

Metadata Sales Data Warehouse Optimization 106

AWS Big Data

JUNE 23, 2025

We are constantly working to improve and evolve it, supported by a large and talented team of software engineers. With deep expertise in data strategy, data warehousing, and big data systems, she helps organizations transform their data landscapes.

Data Warehouse 110

Data Warehouse Metadata Data-driven Reporting 110

AWS Big Data

JUNE 26, 2025

Noritaka Sekiyama is a Principal Big Data Architect with Amazon Web Services (AWS) Analytics services. He’s responsible for building software artifacts to help customers. Abhay Joshi is a Software Development Engineer at AWS Glue and AWS Lake Formation. In his spare time, he enjoys cycling on his road bike.

Metrics 77

Metrics Statistics Dashboards Metadata 77

AWS Big Data

DECEMBER 4, 2024

Open table formats are emerging in the rapidly evolving domain of big data management, fundamentally altering the landscape of data storage and analysis. He is particularly passionate about big data technologies and open source software. He is responsible for building software artifacts to help customers.

Snapshot 84

Snapshot Metadata Data Lake Optimization 84

AWS Big Data

JULY 22, 2025

This includes access to AWS Glue job statuses, Amazon Athena query results, Amazon EMR cluster metrics, and AWS Glue Data Catalog metadata through a unified interface that LLMs can understand and reason about. Arun A K is a Big Data Solutions Architect with AWS. In his free time, Arun loves to enjoy quality time with his family.

Data Processing 55

Data Processing Metadata Interactive Data-driven 55

AWS Big Data

NOVEMBER 22, 2024

About the Authors Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He is responsible for building software artifacts to help customers. Vishal Kajjam is a Software Development Engineer on the AWS Glue team. Wei Tang is a Software Development Engineer on the AWS Glue team.

Metrics 115

Metrics Data Lake Software Optimization 115

AWS Big Data

NOVEMBER 29, 2024

As organizations increasingly become more data driven, this SAP connector can provide an efficient, cost effective, performant, secure way to include SAP source data in your big data and analytic outcomes. For more information see AWS Glue. About the authors Allison Quinn is a Sr.

Visualization 105

Visualization Data Processing Data-driven Cost-Benefit 105

AWS Big Data

NOVEMBER 26, 2024

In March 2024, the project was donated to the Apache Software Foundation (ASF) and rebranded as Apache XTable, where it is now incubating. Then XTable translates between source and target formats and writes the new metadata on the same data store. His areas of interest are data platforms and cloud-native software engineering.

Metadata 106

Metadata Data Lake Snapshot Data Warehouse 106

Smart Data Collective

JUNE 11, 2025

Reading: Monitoring Data Without Turning into Big Brother Share Notification Font Resizer Aa Font Resizer Aa Search About Help Privacy Follow US © 2008-23 SmartData Collective. Alex Blackwell 10 Min Read AI-Generated Image from Google Labs SHARE Data monitoring is no longer optional for large-scale digital systems.

Big Data 68

Big Data Metrics Dashboards Software 68

AWS Big Data

NOVEMBER 8, 2024

He has worked with building data warehouses and big data solutions for over 15+ years. He has over 13 years of professional experience building and optimizing enterprise data warehouses and is passionate about enabling customers to realize the power of their data. Outside of work, he enjoys traveling and cooking.

Data Lake 106

Data Lake Data Warehouse Optimization Testing 106

AWS Big Data

NOVEMBER 22, 2024

Andre Kurait is a Software Development Engineer II at Amazon Web Services, based in Austin, Texas. He has functional domain expertise in distributed systems, AI/ML, cloud-native design, and optimizing DevOps workflows. In his free time, he loves to explore specialty coffee and run through the West Austin hills.

Snapshot 97

Snapshot Metadata Recreation/Entertainment Data Processing 97

AWS Big Data

NOVEMBER 27, 2024

Together with price-performance, Amazon Redshift offers capabilities such as serverless architecture, machine learning integration within your data warehouse and secure data sharing across the organization. dbt Cloud is a hosted service that helps data teams productionize dbt deployments.

Data Warehouse 110

Data Warehouse Analytics Testing Sales 110

AWS Big Data

APRIL 9, 2025

Rajat Mathur is a Software Development Manager at AWS, leading the Amazon DataZone and SageMaker Unified Studio engineering teams. His team designs, builds, and operates services which make it faster and easier for customers to catalog, discover, share, and govern data.

Metadata 109

Metadata Metrics Data-driven Cost-Benefit 109

AWS Big Data

DECEMBER 9, 2024

This streamlined method not only makes the implementation of SCD Type-2 more straightforward, but also offers improved performance and scalability for handling large volumes of historical data in CDC scenarios. Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He works based in Tokyo, Japan.

Snapshot 65

Snapshot Data Warehouse Data Lake Data Quality 65

Smart Data Collective

JUNE 26, 2025

Reading: Stopping Lateral Movement in a Data-Heavy, Edge-First World Share Notification Font Resizer Aa Font Resizer Aa Search About Help Privacy Follow US © 2008-23 SmartData Collective. You must understand how big data can expose new vulnerabilities if not properly secured. All Rights Reserved. All Rights Reserved.

Big Data 71

Big Data IoT Data-driven Business Intelligence 71

CIO Business Intelligence

FEBRUARY 20, 2025

Outdated software applications are creating roadblocks to AI adoption at many organizations, with limited data retention capabilities a central culprit, IT experts say. Moreover, the cost of maintaining outdated software, with a shrinking number of software engineers familiar with the apps, can be expensive, he says.

Insurance 108

Insurance Cost-Benefit Unstructured Data Data Lake 108

AWS Big Data

JULY 15, 2025

About the authors Chiho Sugimoto is a Cloud Support Engineer on the AWS Big Data Support team. She is passionate about helping customers build data lakes using ETL workloads. Noritaka Sekiyama is a Principal Big Data Architect at the AWS Analytics product team. In his spare time, he enjoys cycling on his road bike.

Visualization 86

Visualization Data Processing Metrics Big Data 86

AWS Big Data

OCTOBER 30, 2024

Eric Fleishman is a software engineer at AWS in Seattle. He’s worked with small and big data for most of his career, and has built applications running on AWS since 2008. She focuses on crafting cloud-based data platforms, enabling real-time streaming, big data processing, and robust data governance.

Visualization 120

Visualization Data Lake Testing Data Governance 120

AWS Big Data

JULY 23, 2025

He leads generative AI feature development across services such as AWS Glue, Amazon EMR, and Amazon MWAA, using AI/ML to simplify and enhance the experience of data practitioners building data applications on AWS. Kartik Panjabi is a Software Development Manager on the AWS Glue team.

Optimization 81

Optimization Metrics Data-driven Data Integration 81

AWS Big Data

JULY 15, 2025

Noritaka Sekiyama is a Principal Big Data Architect with AWS Analytics services. He’s responsible for building software artifacts to help customers. Regan Baum is a Senior Software Development Engineer on the Amazon SageMaker Unified Studio team. He supports customers resolve technical issues and launch systems.

Data Processing 96

Data Processing Visualization Metadata Software 96

AWS Big Data

JULY 25, 2025

As a result, organizations collect vast amounts of data from diverse sensor devices monitoring everything from industrial equipment to smart buildings. These sensor devices frequently undergo firmware updates, software modifications, or configuration changes that introduce new monitoring capabilities or retire obsolete metrics.

IoT 85

IoT Analytics Metadata Measurement 85