Ontotext

NOVEMBER 18, 2021

And all of them are asking hard questions: “Can you integrate my data, with my particular format?”, “How well can you scale?”, “How many visualizations do you offer?”. Nowadays, data analytics doesn’t exist on its own. You have to take care of data extraction, transformation and loading, and of visualization.

Visualization 98

Visualization Reporting Metadata Enterprise 98

AWS Big Data

DECEMBER 13, 2023

In addition to using native managed AWS services that BMS didn’t need to worry about upgrading, BMS was looking to offer an ETL service to non-technical business users that could visually compose data transformation workflows and seamlessly run them on the AWS Glue Apache Spark-based serverless data integration engine.

Metadata 101

Metadata Data Lake Visualization Data Quality 101

Wayne Yaddow

FEBRUARY 18, 2025

Selecting the strategies and tools for validating data transformations and data conversions in your data pipelines. Introduction Data transformations and data conversions are crucial to ensure that raw data is organized, processed, and ready for useful analysis.

Testing 52

Testing Data Transformation Data-driven Data Quality 52

Wayne Yaddow

FEBRUARY 26, 2025

AI is transforming how senior data engineers and data scientists validate data transformations and conversions. Artificial intelligence-based verification approaches aid in the detection of anomalies, the enforcement of data integrity, and the optimization of pipelines for improved efficiency.

Data Transformation 52

Data Transformation Testing Data-driven Data Quality 52

erwin

APRIL 29, 2020

These tools range from enterprise service bus (ESB) products, data integration tools; extract, transform and load (ETL) tools, procedural code, application program interfaces (API)s, file transfer protocol (FTP) processes, and even business intelligence (BI) reports that further aggregate and transform data. Data Quality.

Metadata 111

Metadata Key Performance Indicator Data Governance Data Quality 111

Wayne Yaddow

MARCH 5, 2025

In this post, well see the fundamental procedures, tools, and techniques that data engineers, data scientists, and QA/testing teams use to ensure high-quality data as soon as its deployed. First, we look at how unit and integration tests uncover transformation errors at an early stage. Statistical tests (e.g.,

Testing 52

Testing Data Transformation Statistics Metadata 52

AWS Big Data

DECEMBER 21, 2023

AWS Glue is a serverless data integration service that makes it straightforward to discover, prepare, and combine data for analytics, machine learning (ML), and application development. AWS Glue provides both visual and code-based interfaces to make data integration effortless. Choose Create job and Visual ETL.

Analytics 107

Analytics IT Data Lake Visualization 107

Sisense

OCTOBER 15, 2020

Every data professional knows that ensuring data quality is vital to producing usable query results. Streaming data can be extra challenging in this regard, as it tends to be “dirty,” with new fields that are added without warning and frequent mistakes in the data collection process.

Dashboards 98

Dashboards IoT Optimization Internet of Things 98

AWS Big Data

AUGUST 1, 2024

However, you might face significant challenges when planning for a large-scale data warehouse migration. Data engineers are crucial for schema conversion and data transformation, and DBAs can handle cluster configuration and workload monitoring. Platform architects define a well-architected platform.

Data Warehouse 99

Data Warehouse KPI Optimization Cost-Benefit 99

AWS Big Data

FEBRUARY 8, 2023

It’s common to ingest multiple data sources into Amazon Redshift to perform analytics. Often, each data source will have its own processes of creating and maintaining data, which can lead to data quality challenges within and across sources. Answering questions as simple as “How many unique customers do we have?”

Data Quality 84

Data Quality Testing Data Warehouse Unstructured Data 84

Octopai

AUGUST 11, 2024

Just as a navigation app provides a detailed map of roads, guiding you from your starting point to your destination while highlighting every turn and intersection, data flow lineage offers a comprehensive view of data movement and transformations throughout its lifecycle. Predefined type for various data sources.

Metadata 40

Metadata Visualization Data Quality Data Governance 40

Octopai

JANUARY 19, 2021

With Octopai’s support and analysis of Azure Data Factory, enterprises can now view complete end-to-end data lineage from Azure Data Factory all the way through to reporting for the first time ever.

Metadata 52

Metadata ROI Machine Learning Data Quality 52

AWS Big Data

APRIL 5, 2023

In this post, we share how Showpad used QuickSight to streamline data and insights access across teams and customers. Showpad migrated over 70 dashboards with over 1,000 visuals. Each of these tools were getting data from a different place, and that’s where it gets difficult,” says Jeroen Minnaert, head of data at Showpad. “If

Dashboards 52

Dashboards Reporting Cost-Benefit Visualization 52

AWS Big Data

MARCH 3, 2023

Every event in the data source can be relevant, and our customers don’t tolerate data loss, poor data quality, or discrepancies between the source and Tricentis Analytics. While aggregating, summarizing, and aligning to a common information model, all transformations must not affect the integrity of data from its source.

Software 52

Software Data Lake Testing Cost-Benefit 52

Jet Global

MAY 9, 2024

Data Extraction : The process of gathering data from disparate sources, each of which may have its own schema defining the structure and format of the data and making it available for processing. This can include tasks such as data ingestion, cleansing, filtering, aggregation, or standardization.

Data Lake 52

Data Lake Data Warehouse Business Intelligence Machine Learning 52

Jet Global

FEBRUARY 23, 2024

The quick and dirty definition of data mapping is the process of connecting different types of data from various data sources. Data mapping is a crucial step in data modeling and can help organizations achieve their business goals by enabling data integration, migration, transformation, and quality.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Visualization 52

Jet Global

MARCH 13, 2024

Complex Data Structures and Integration Processes Dynamics data structures are already complex – finance teams navigating Dynamics data frequently require IT department support to complete their routine reporting.

Reporting 52

Reporting Finance Data Quality Sales 52

Jet Global

MARCH 14, 2024

Jet’s interface lets you handle data administration easily, without advanced coding skills. You don’t need technical skills to manage complex data workflows in the Fabric environment. Data Lineage and Documentation Jet Analytics simplifies the process of documenting data assets and tracking data lineage in Fabric.

Analytics 52

Analytics Management Reporting Data Quality 52

AWS Big Data

DECEMBER 17, 2024

Prevent the inclusion of invalid values in categorical data and process data without any data loss. Conduct data quality tests on anonymized data in compliance with data policies Conduct data quality tests to quickly identify and address data quality issues, maintaining high-quality data at all times.

Data Quality 69

Data Quality Testing Metrics Optimization 69

BI-Survey

MARCH 6, 2025

For data management teams, achieving more with fewer resources has become a familiar challenge. While efficiency is a priority, data quality and security remain non-negotiable. Developing and maintaining data transformation pipelines are among the first tasks to be targeted for automation.

Data Warehouse 52

Data Warehouse Metadata Unstructured Data Data-driven 52

DataKitchen

JANUARY 23, 2023

It allows organizations to see how data is being used, where it is coming from, its quality, and how it is being transformed. DataOps Observability includes monitoring and testing the data pipeline, data quality, data testing, and alerting. Data lineage does not directly improve data quality.

Testing 130

Testing Data Governance Data Quality Data-driven 130