AWS Big Data

AUGUST 9, 2024

Dafiti’s data infrastructure relies heavily on ETL and ELT processes, with approximately 2,500 unique processes run daily. Amazon Redshift at Dafiti Amazon Redshift is a fully managed data warehouse service, and was adopted by Dafiti in 2017. Do you want to know more about what we’re doing in the data area at Dafiti?

Data Lake 111

Data Lake Analytics Data Warehouse Data-driven 111

AWS Big Data

DECEMBER 21, 2023

To speed up the self-service analytics and foster innovation based on data, a solution was needed to provide ways to allow any team to create data products on their own in a decentralized manner. To create and manage the data products, smava uses Amazon Redshift , a cloud data warehouse.

Data Lake 115

Data Lake Data Warehouse Data-driven B2B 115

AWS Big Data

NOVEMBER 15, 2023

Once released, consumers use datasets from different providers for analysis, machine learning (ML) workloads, and visualization. Each CDH dataset has three processing layers: source (raw data), prepared (transformed data in Parquet), and semantic (combined datasets).

Analytics 111

Analytics Dashboards Metadata Data Warehouse 111

Cloudera

AUGUST 8, 2022

The general availability covers Iceberg running within some of the key data services in CDP, including Cloudera Data Warehouse ( CDW ), Cloudera Data Engineering ( CDE ), and Cloudera Machine Learning ( CML ). Cloudera Data Engineering (Spark 3) with Airflow enabled. Cloudera Machine Learning .

Snapshot 94

Snapshot Data Warehouse Machine Learning Cost-Benefit 94

Alation

JANUARY 17, 2023

The modern data stack is a combination of various software tools used to collect, process, and store data on a well-integrated cloud-based data platform. It is known to have benefits in handling data due to its robustness, speed, and scalability. A typical modern data stack consists of the following: A data warehouse.

Data Warehouse 52

Data Warehouse Cost-Benefit Data Science Data Transformation 52

AWS Big Data

FEBRUARY 8, 2023

Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. Amazon Redshift enables you to run complex SQL analytics at scale and performance on terabytes to petabytes of structured and unstructured data, and make the insights widely available through popular business intelligence (BI) and analytics tools.

Data Quality 98

Data Quality Testing Data Warehouse Unstructured Data 98

AWS Big Data

JUNE 29, 2023

In this post, we discuss why AWS recommends moving from Kinesis Data Analytics for SQL Applications to Amazon Kinesis Data Analytics for Apache Flink to take advantage of Apache Flink’s advanced streaming capabilities. Notebooks are provisioned quickly and provide a way for you to instantly view and analyze your streaming data.

Data Analytics 104

Data Analytics Analytics IoT Data Lake 104

IBM Big Data Hub

JULY 17, 2023

By supporting open-source frameworks and tools for code-based, automated and visual data science capabilities — all in a secure, trusted studio environment — we’re already seeing excitement from companies ready to use both foundation models and machine learning to accomplish key tasks.

Machine Learning 62

Machine Learning Data Warehouse Modeling Cost-Benefit 62

AWS Big Data

MARCH 3, 2023

Tricentis is the global leader in continuous testing for DevOps, cloud, and enterprise applications. Speed changes everything, and continuous testing across the entire CI/CD lifecycle is the key. Tricentis instills that confidence by providing software tools that enable Agile Continuous Testing (ACT) at scale.

Software 52

Software Data Lake Testing Cost-Benefit 52

Alation

OCTOBER 27, 2022

Few actors in the modern data stack have inspired the enthusiasm and fervent support as dbt. This data transformation tool enables data analysts and engineers to transform, test and document data in the cloud data warehouse. But what does this mean from a practitioner perspective?

Dashboards 52

Dashboards Metrics Sales Reporting 52

AWS Big Data

JULY 31, 2023

This solution decouples the ETL and analytics workloads from our transactional data source Amazon Aurora, and uses Amazon Redshift as the data warehouse solution to build a data mart. We use Amazon Redshift as the data warehouse to implement the data mart solution. Navigate to the Visual tab.

Sales 52

Sales Data Warehouse Visualization Testing 52

Occam's Razor

OCTOBER 19, 2010

If after rigorous analysis you have determined that you have evolved to a stage that you need a data warehouse then you are out of luck with Yahoo! If you can show ROI on a DW it would be a good use of your money to go with Omniture Discover, WebTrends Data Mart, Coremetrics Explore. Five Reasons And Awesome Testing Ideas.

Analytics 136

Analytics Testing Measurement Optimization 136

Jet Global

MAY 1, 2023

This is in contrast to traditional BI, which extracts insight from data outside of the app. We rely on increasingly mobile technology to comb through massive amounts of data and solve high-value problems. Plus, there is an expectation that tools be visually appealing to boot. Their dashboards were visually stunning.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52

Jet Global

FEBRUARY 23, 2024

Why Data Mapping is Important Data mapping is a critical element of any data management initiative, such as data integration, data migration, data transformation, data warehousing, or automation. Data mapping is important for several reasons.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Visualization 52

Jet Global

JULY 16, 2024

By providing a consistent and stable backend, Apache Iceberg ensures that data remains immutable and query performance is optimized, thus enabling businesses to trust and rely on their BI tools for critical insights. It provides a stable schema, supports complex data transformations, and ensures atomic operations.

Dashboards 52

Dashboards Data-driven Reporting Business Intelligence 52

AWS Big Data

DECEMBER 17, 2024

Conduct data quality tests on anonymized data in compliance with data policies Conduct data quality tests to quickly identify and address data quality issues, maintaining high-quality data at all times. The challenge Data quality tests require performing 1,300 tests on 10 TB of data monthly.

Data Quality 91

Data Quality Testing Metrics Optimization 91

BI-Survey

MARCH 6, 2025

While efficiency is a priority, data quality and security remain non-negotiable. Developing and maintaining data transformation pipelines are among the first tasks to be targeted for automation. However, caution is advised since accuracy, timeliness, and other aspects of data quality depend on the quality of data pipelines.

Data Warehouse 52

Data Warehouse Metadata Unstructured Data Data-driven 52

AWS Big Data

APRIL 29, 2025

While enabling organization-wide efficiency, the team also applied these principles to the data architecture, making sure that CLEA itself operates frugally. After evaluating various tools, we built a serverless data transformation pipeline using Amazon Athena and dbt. However, our initial data architecture led to challenges.

Data Transformation 54

Data Transformation Cost-Benefit Testing Data Lake 54