Smart Data Collective

JULY 16, 2025

Reading: Turning Data Into Decisions: How Analytics Improves Transportation Strategy Share Notification Font Resizer Aa Font Resizer Aa Search About Help Privacy Follow US © 2008-23 SmartData Collective. Andrej Kovacevic 3 Min Read Licensed Photo from Microsoft Stock Images SHARE Transportation networks generate a constant stream of data.

Strategy

David Menninger's Analyst Perspectives

JULY 1, 2025

If a single phrase could sum up the big data craze of a dozen or so years ago, it would be “more data beats better algorithms.” The phrase was, of course, an oversimplification, and enterprises investing in big data projects quickly found that quantity was not the only characteristic of data that mattered.

Management

AWS Big Data

JANUARY 6, 2025

He has helped customers build scalable data warehousing and big data solutions for over 16 years. He has worked with building data warehouses and big data solutions for over 13 years. He loves to design and build efficient end-to-end solutions on AWS. Tahir Aziz is an Analytics Solution Architect at AWS.

Analytics

Analytics Vidhya

NOVEMBER 22, 2024

In the era of big data and rapid technological advancement, the ability to analyze and interpret data effectively has become a cornerstone of decision-making and innovation. Python, renowned for its simplicity and versatility, has emerged as the leading programming language for data analysis.

Big Data

Smart Data Collective

JULY 20, 2025

More Read 5 Reasons Data-Savvy Accountants Are Becoming Vital to Businesses Here Are The Most Important Ways To Ensure Customer Data Protection Blasphemy? The growing need for big data is another. All Rights Reserved. There are over 1,897,100 software engineers in the U.S. Followers Like 33.7k Followers Like 33.7k

Big Data

AWS Big Data

APRIL 8, 2025

He is particularly passionate about big data technologies and open source software. Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He supports customers across a wide range of industries in building and operating analytics platforms more effectively. He works based in Tokyo, Japan.

Snapshot

AWS Big Data

NOVEMBER 22, 2024

About the Authors Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. Keerthi Chadalavada is a Senior Software Development Engineer at AWS Glue, focusing on combining generative AI and data integration technologies to design and build comprehensive solutions for customers’ data and analytics needs.

Cost-Benefit

AWS Big Data

NOVEMBER 14, 2024

The landscape of big data management has been transformed by the rising popularity of open table formats such as Apache Iceberg, Apache Hudi, and Linux Foundation Delta Lake. These formats, designed to address the limitations of traditional data storage systems, have become essential in modern data architectures.

Metadata

AWS Big Data

NOVEMBER 21, 2024

Amazon EMR is a cloud big data platform for petabyte-scale data processing, interactive analysis, streaming, and machine learning (ML) using open source frameworks such as Apache Spark , Presto and Trino , and Apache Flink. About the Authors Garima Arora is a Software Development Engineer for Amazon EMR at Amazon Web Services.

Metrics

AWS Big Data

DECEMBER 4, 2024

About the Authors Praveen Kumar is an Analytics Solutions Architect at AWS with expertise in designing, building, and implementing modern data and analytics platforms using cloud-based services. His areas of interest are serverless technology, data governance, and data-driven AI applications.

Visualization

AWS Big Data

OCTOBER 30, 2024

This is part two of a three-part series where we show how to build a data lake on AWS using a modern data architecture. This post shows how to load data from a legacy database (SQL Server) into a transactional data lake ( Apache Iceberg ) using AWS Glue.

Data Lake

Smart Data Collective

JUNE 16, 2025

He is passionate about covering topics like big data, business intelligence, startups & entrepreneurship. Artificial Intelligence for eCommerce: A Closer Look Artificial Intelligence How To Get An Award Winning Giveaway Bot Big Data Chatbots Exclusive Quick Link About Contact Privacy Follow US © 2008-25 SmartData Collective.

Business Intelligence

AWS Big Data

DECEMBER 12, 2024

Has many years of experience in big data, enterprise digital transformation research and development, consulting, and project management across telecommunications, entertainment, and financial industries.

Snapshot

AWS Big Data

JANUARY 15, 2025

Lakshmi Nair is a Senior Specialist Solutions Architect for Data Analytics at AWS. She focuses on architecting solutions for organizations across their end-to-end data analytics estate, including batch and real-time streaming, data governance, big data, data warehousing, and data lake workloads.

IoT

AWS Big Data

OCTOBER 30, 2024

Users can begin ingesting data to Redshift from Amazon S3 with simple SQL commands and gain access to the most up-to-date data without the need for third-party tools or custom implementation. He has worked with building data warehouses and big data solutions for over 15+ years.

Data Warehouse

AWS Big Data

NOVEMBER 7, 2024

He has helped customers build scalable data warehousing and big data solutions for over 16 years. About the authors Ritesh Kumar Sinha is an Analytics Specialist Solutions Architect based out of San Francisco. He loves to design and build efficient end-to-end solutions on AWS.

Data Warehouse

AWS Big Data

DECEMBER 16, 2024

About the Authors Chiho Sugimoto is a Cloud Support Engineer on the AWS Big Data Support team. She is passionate about helping customers build data lakes using ETL workloads. Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team.

Visualization

CIO Business Intelligence

DECEMBER 11, 2024

As the chief architect and head of data engineering at Equifaxs USIS business unit, he drove the technology strategy and ran a large data engineering organization to completely transform the company. He is currently a technology advisor to multiple startups and mid-size companies.

Management

DataFloq

JULY 15, 2025

Their guidance encourages financial institutions to adopt advanced analytics, real time decisioning, and data pooling to manage risk at scale. A recent study outlines how big data systems benefit from contextual decision making, mirroring what’s needed in financial crime compliance.

Machine Learning

AWS Big Data

DECEMBER 4, 2024

SageMaker brings together widely adopted AWS ML and analytics capabilities—virtually all of the components you need for data exploration, preparation, and integration; petabyte-scale big data processing; fast SQL analytics; model development and training; governance; and generative AI development.

Data Analytics

AWS Big Data

JULY 25, 2025

EMR Serverless makes running big data analytics frameworks straightforward by offering a serverless option that automatically provisions and manages the infrastructure required to run big data applications. Venkat is a Technology Strategy Leader in Data, AI, ML, generative AI, and Advanced Analytics.

Interactive

AWS Big Data

JUNE 18, 2025

We encourage you to evaluate RocksDB for your use cases, particularly if you’re experiencing memory pressure issues with the default state store or need to handle large amounts of state data in your streaming applications. About the authors Melody Yang is a Senior Big Data Solution Architect for Amazon EMR at AWS.

Optimization

AWS Big Data

OCTOBER 23, 2024

Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is a fully managed service that builds upon Apache Airflow, offering its benefits while eliminating the need for you to set up, operate, and maintain the underlying infrastructure, reducing operational overhead while increasing security and resilience.

Interactive

AWS Big Data

NOVEMBER 22, 2024

About the Authors Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. He is responsible for building software artifacts to help customers. In his spare time, he enjoys cycling with his road bike. Vishal Kajjam is a Software Development Engineer on the AWS Glue team.

Metrics

CIO Business Intelligence

AUGUST 4, 2025

I don’t have to move around to many different platforms and technologies because I have everything in one place; I can do SQL, I can do big data with trillions of rows, I can do fast queries, and all of the LLMs run natively there.” “It’s like this one-stop shop,” he says. “I

Enterprise

AWS Big Data

DECEMBER 27, 2024

To stay informed, subscribe to the AWS Big Data Blogs RSS feed , where you can find updates on the EMR runtime for Spark and Iceberg, as well as tips on configuration best practices and tuning recommendations. This is a further increase of 32% from EMR 7.1.

Cost-Benefit

AWS Big Data

OCTOBER 25, 2024

Analytics Specialist Solutions Architect at Amazon Web Services (AWS) Philippines, specializing in big data and analytics. She helps customers in designing and implementing scalable, secure, and cost-effective data solutions, as well as migrating and modernizing their big data and analytics workloads to AWS.

Metrics

AWS Big Data

JUNE 6, 2025

About the authors Narayani Ambashta is an Analytics Specialist Solutions Architect at AWS, focusing on the automotive and manufacturing sector, where she guides strategic customers in developing modern data and AI strategies. He helps customers architect and build highly scalable, performant, and secure cloud-based solutions on AWS.

Analytics

AWS Big Data

OCTOBER 11, 2024

Snapshots are crucial for data backup and disaster recovery in Amazon OpenSearch Service. Snapshots play a critical role in providing the availability, integrity and ability to recover data in OpenSearch Service domains.

Snapshot

AWS Big Data

NOVEMBER 7, 2024

Amazon Redshift is a fully managed, AI-powered cloud data warehouse that delivers the best price-performance for your analytics workloads at any scale. Amazon Q generative SQL brings the capabilities of generative AI directly into the Amazon Redshift query editor.

Metadata

AWS Big Data

DECEMBER 4, 2024

About the Authors Noritaka Sekiyama is a Principal Big Data Architect on the AWS Glue team. Stuti Deshpande is a Big Data Specialist Solutions Architect at AWS. She has extensive experience in big data, ETL, and analytics. He is responsible for building software artifacts to help customers.

Data Lake

DataFloq

JULY 25, 2025

The area of MLOps has become much more than a buzzword-it is very much a fundamental part of AI deployment today. It is projected that the global MLOps market will reach USD 3.03 billion in 2025, representing an increase from USD 2.19 billion in 2024 and a CAGR of 40.5% for 2025-2030, according to a report from Grand View Research.

Machine Learning

AWS Big Data

DECEMBER 19, 2024

Organizational data is often fragmented across multiple lines of business, leading to inconsistent and sometimes duplicate datasets. This fragmentation can delay decision-making and erode trust in available data.

Publishing

DataFloq

JULY 8, 2025

As organizations race to adopt generative AI tools-from AI writing assistants to autonomous coding platforms-one often-overlooked variable makes the difference between game-changing innovation and disastrous missteps: data quality. It consumes data, learns from it, and produces outcomes that reflect the quality of what it was trained on.

Data Quality

AWS Big Data

JANUARY 9, 2025

In this post, we focus on data management implementation options such as accessing data directly in Amazon Simple Storage Service (Amazon S3), using popular data formats like Parquet, or using open table formats like Iceberg.

Metadata