KDnuggets

SEPTEMBER 23, 2024

It's important to transform data for effective data analysis. R's 'dplyr' package makes data transformation simple and efficient. This article will teach you how to use the dplyr package for data transformation in R. Install dplyr Before using dplyr, you must install and load it into your R session.

Data Transformation 116

Data Transformation IT 116

CIO Business Intelligence

OCTOBER 18, 2022

The old stadium, which opened in 1992, provided the business operations team with data, but that data came from disparate sources, many of which were not consistently updated. In 2016, Major League Baseball’s Texas Rangers announced it would build a brand-new state-of-the-art stadium in Arlington, Texas.

Data Transformation 141

Data Transformation Consulting Data Lake Reporting 141

Analytics Vidhya

JUNE 14, 2024

Introduction Have you ever struggled with managing complex data transformations? In today’s data-driven world, extracting, transforming, and loading (ETL) data is crucial for gaining valuable insights. While many ETL tools exist, dbt (data build tool) is emerging as a game-changer.

Data-driven 306

Data-driven Data Transformation Management Analytics 306

Analytics Vidhya

APRIL 29, 2022

Introduction Azure data factory (ADF) is a cloud-based ETL (Extract, Transform, Load) tool and data integration service which allows you to create a data-driven workflow. The data-driven workflow in ADF orchestrates and automates the data movement and data transformation.

Data-driven 328

Data-driven Data Science Data Transformation Data Integration 328

CIO Business Intelligence

AUGUST 9, 2024

At Atlanta’s Hartsfield-Jackson International Airport, an IT pilot has led to a wholesale data journey destined to transform operations at the world’s busiest airport, fueled by machine learning and generative AI.

Data Transformation 126

Data Transformation Machine Learning Data Lake Dashboards 126

Smart Data Collective

JULY 25, 2019

The post Is Big Data Transforming Our Broken Hospital Management Systems? But regardless of what you pick, one thing is certain — it is essential to have an effective software product for a hospital to manage various health care and administrative tasks. appeared first on SmartData Collective.

Big Data 110

Big Data Data Transformation Management Software 110

Analytics Vidhya

JUNE 18, 2019

Overview The Transformer model in NLP has truly changed the way we work with text data Transformer is behind the recent NLP developments, including. The post How do Transformers Work in NLP? A Guide to the Latest State-of-the-Art Models appeared first on Analytics Vidhya.

Modeling 307

Modeling Data Transformation Analytics Deep Learning 307

AWS Big Data

NOVEMBER 22, 2024

The need for streamlined data transformations As organizations increasingly adopt cloud-based data lakes and warehouses, the demand for efficient data transformation tools has grown. This approach helps in managing storage costs while maintaining the flexibility to analyze historical trends when needed.

Data Lake 101

Data Lake Data Warehouse Cost-Benefit Data Transformation 101

KDnuggets

OCTOBER 22, 2024

Utilize these transformation techniques in your data workflow.

Data Transformation 123

Data Transformation 123

Analytics Vidhya

FEBRUARY 4, 2023

Introduction This article will explain the difference between ETL (Extract, Transform, Load) and ELT (Extract, Load, Transform) when data transformation occurs. In ETL, data is extracted from multiple locations to meet the requirements of the target data file and then placed into the file.

Data Transformation 298

Data Transformation Analytics Data Warehouse Data Integration 298

Analytics Vidhya

AUGUST 29, 2022

Introduction Apache Flink is a big data framework that allows programmers to process huge amounts of data in a very efficient and scalable way. This article will introduce some basic API concepts and standard data transformations available in the Apache Flink Java API. The […].

Big Data 291

Big Data Data Science Data Transformation Publishing 291

Analytics Vidhya

FEBRUARY 20, 2023

Introduction Azure data factory (ADF) is a cloud-based data ingestion and ETL (Extract, Transform, Load) tool. The data-driven workflow in ADF orchestrates and automates data movement and data transformation.

Data-driven 283

Data-driven Data Transformation Analytics Data Warehouse 283

Bruno Aziza

JULY 1, 2020

Leading companies are undertaking a wholesale transformation to enable remote work leveraging digital capabilities and data.

Data Transformation 125

Data Transformation Technology 125

DataCamp

SEPTEMBER 27, 2021

We’ve compiled a list of resources to inform your data transformation, data culture initiative, and data upskilling. Explore these webinars and white papers.

Data Transformation 90

Data Transformation 90

Wayne Yaddow

MARCH 14, 2025

How dbt Core aids data teams test, validate, and monitor complex data transformations and conversions Photo by NASA on Unsplash Introduction dbt Core, an open-source framework for developing, testing, and documenting SQL-based data transformations, has become a must-have tool for modern data teams as the complexity of data pipelines grows.

Data Transformation 52

Data Transformation Testing Unstructured Data Data Quality 52

Wayne Yaddow

MARCH 12, 2025

Data quality rules are codified into structured Expectation Suites by Great Expectations instead of relying on ad-hoc scripts or manual checks. The framework ensures that your data transformations comply with rigorous specifications from the moment they are created through every iteration of your pipeline.

Data Transformation 52

Data Transformation Data Quality Testing Data Warehouse 52

Analytics Vidhya

DECEMBER 11, 2023

Introduction Power Query is a powerful data transformation and manipulation tool in PowerBI that allows users to extract, transform, and load data from various sources. It provides a user-friendly interface for performing complex data transformations without the need for coding.

Data Transformation 178

Data Transformation Analytics IT 178

Wayne Yaddow

FEBRUARY 26, 2025

AI is transforming how senior data engineers and data scientists validate data transformations and conversions. Artificial intelligence-based verification approaches aid in the detection of anomalies, the enforcement of data integrity, and the optimization of pipelines for improved efficiency.

Data Transformation 52

Data Transformation Testing Data-driven Data Quality 52

Wayne Yaddow

FEBRUARY 21, 2025

Complex Data TransformationsTest Planning Best Practices Ensuring data accuracy with structured testing and best practices Photo by Taylor Vick on Unsplash Introduction Data transformations and conversions are crucial for data pipelines, enabling organizations to process, integrate, and refine raw data into meaningful insights.

Testing 52

Testing Data Transformation Data Quality Data Integration 52

Wayne Yaddow

FEBRUARY 4, 2025

Its EssentialVerifying Data Transformations (Part4) Uncovering the leading problems in data transformation workflowsand practical ways to detect and preventthem In Parts 13 of this series of blogs, categories of data transformations were identified as among the top causes of data quality defects in data pipeline workflows.

Data Transformation 52

Data Transformation Testing Data Quality Strategy 52

Wayne Yaddow

FEBRUARY 18, 2025

Selecting the strategies and tools for validating data transformations and data conversions in your data pipelines. Introduction Data transformations and data conversions are crucial to ensure that raw data is organized, processed, and ready for useful analysis.

Testing 52

Testing Data Transformation Data-driven Data Quality 52

Wayne Yaddow

FEBRUARY 6, 2025

Common challenges and practical mitigation strategies for reliable data transformations. Photo by Mika Baumeister on Unsplash Introduction Data transformations are important processes in data engineering, enabling organizations to structure, enrich, and integrate data for analytics , reporting, and operational decision-making.

Testing 52

Testing Data Transformation Data-driven Manufacturing 52

Wayne Yaddow

FEBRUARY 11, 2025

Managing tests of complex data transformations when automated data testing tools lack important features? Photo by Marvin Meyer on Unsplash Introduction Data transformations are at the core of modern business intelligence, blending and converting disparate datasets into coherent, reliable outputs.

Testing 52

Testing Data Transformation Data Quality Statistics 52

Wayne Yaddow

MARCH 5, 2025

In this post, well see the fundamental procedures, tools, and techniques that data engineers, data scientists, and QA/testing teams use to ensure high-quality data as soon as its deployed. First, we look at how unit and integration tests uncover transformation errors at an early stage.

Testing 52

Testing Data Transformation Statistics Metadata 52

TDAN

AUGUST 15, 2023

Nearly every data leader I talk to is in the midst of a data transformation. As businesses look for ways to increase sales, improve customer experience, and stay ahead of the competition, they are realizing that data is their competitive advantage and the key to achieving their goals. And it’s no surprise, really.

Data Transformation 52

Data Transformation Sales Metadata Data Governance 52

Bruno Aziza

AUGUST 18, 2019

Although CIO’s and CDO’s aspire to be on the offensive in using data to drive revenue generation and business growth, it is defensive initiatives that are providing cover for forward-looking transformation ambitions.

Data Transformation 77

Data Transformation IT 77

Dataiku

SEPTEMBER 26, 2022

Whether it’s being used as part of an Extract Transform Load process in data warehousing (or in some cases with modern cloud data stacks , ELT), for use in ML models, or simply ad hoc analysis, data transformation is a key part of nearly any project involving data.

Data Transformation 52

Data Transformation Modeling IT 52

CIO Business Intelligence

DECEMBER 9, 2024

For now, 51% say this strategic alignment has not been fully achieved, according to NTT DATAs study. [3] Data readiness and governance are critical to success and must be addressed in tandem with business process transformation. 3] Preparation. Operations.

Technology 98

Technology Insurance Interactive Reporting 98

Alation

OCTOBER 26, 2021

Building a Data Culture Within a Finance Department. Our finance users tell us that their first exposure to the Alation Data Catalog often comes soon after the launch of organization-wide data transformation efforts. After all, finance is one of the greatest consumers of data within a business.

Finance 52

Finance Data Transformation Enterprise Metrics 52

KDnuggets

OCTOBER 3, 2024

Learn the data engineering tools for data orchestration, database management, batch processing, ETL (Extract, Transform, Load), data transformation, data visualization, and data streaming.

Visualization 138

Visualization Data Transformation Management 138

CIO Business Intelligence

FEBRUARY 27, 2025

Additionally, integrating mainframe data with the cloud enables enterprises to feed information into data lakes and data lake houses, which is ideal for authorized data professionals to easily leverage the best and most modern tools for analytics and forecasting. Four key challenges prevent them from doing so: 1.

Metadata 98

Metadata Data Lake Cost-Benefit Forecasting 98

TDAN

FEBRUARY 21, 2024

In a world increasingly dominated by data, organizations are grappling with the need to effectively manage and harness this valuable asset.

Data Transformation 59

Data Transformation Digital Transformation Data-driven Management 59

DataKitchen

JUNE 30, 2021

Data Transformers podcast hosts Peggy Tsai & Ramesh Dontha chat with DataKitchen CEO Chris Bergh about how DataOps should be 10% of every data team member's job. The post DataOps Should Be Part of Everyone on the Data Team first appeared on DataKitchen.

Data Processing 130

Data Processing Data Transformation 130

MLWhiz

AUGUST 26, 2020

Pandas is one of the best data manipulation libraries in recent times. It lets you slice and dice, groupby, join and do any arbitrary data transformation. You can take a look at this post, which talks about handling most of the data manipulation cases using a straightforward, simple, and matter of fact way using Pandas.

Slice and Dice 131

Slice and Dice Data Transformation IT 131

CIO Business Intelligence

NOVEMBER 17, 2022

Data has become a top priority for businesses large and small, and while some companies have already established a digital strategy, many of them are just getting started. However, the ability to drive digital technology transformation is going to be the focus,” says Stephen Van Vreede, resume expert at IT Tech Exec.

Data Governance 128

Data Governance Data Transformation Strategy Reporting 128

AWS Big Data

DECEMBER 20, 2024

Your generated jobs can use a variety of data transformations, including filters, projections, unions, joins, and aggregations, giving you the flexibility to handle complex data processing requirements. In this post, we discuss how Amazon Q data integration transforms ETL workflow development.

Data Integration 105

Data Integration Visualization Data Processing Big Data 105