datapine

MAY 2, 2023

Benefits Of Big Data In Logistics Before we look at our selection of practical examples and applications, let’s look at the benefits of big data in logistics – starting with the (not so) small matter of costs. A testament to the rising role of optimization in logistics. Why are logistics companies so interested in optimization?

Big Data 275

Big Data Internet of Things Cost-Benefit Optimization 275

DataKitchen

APRIL 26, 2021

In early April 2021, DataKItchen sat down with Jonathan Hodges, VP Data Management & Analytics, at Workiva ; Chuck Smith, VP of R&D Data Strategy at GlaxoSmithKline (GSK) ; and Chris Bergh, CEO and Head Chef at DataKitchen, to find out about their enterprise DataOps transformation journey, including key successes and lessons learned.

Measurement 246

Measurement Metrics Data-driven Dashboards 246

CDW Research Hub

OCTOBER 20, 2020

Key features include: A scalable pipeline to store and process transaction data, supporting daily update updates to a reporting dashboard with high-performance analytics. Manageability and use for non-technical users, democratizing data enterprisewide. Have questions? Contact us.

Dashboards 72

Dashboards Reporting Data Warehouse Structured Data 72

O'Reilly on Data

OCTOBER 19, 2021

If you ask an engineer to show how they operate the application in production, they will likely show containers and operational dashboards—not unlike any other software service. Data is at the core of any ML project, so data infrastructure is a foundational concern. Foundational Infrastructure Layers. Orchestration. Versioning.

IT 364

IT Testing Experimentation Software 364

AWS Big Data

NOVEMBER 22, 2024

Amazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that you can use to analyze your data at scale. Reusing database sessions to simplify the connection management logic in your API implementation, reducing the complexity of the code and making it more straightforward to maintain and scale.

Data Warehouse 92

Data Warehouse Recreation/Entertainment Cost-Benefit Data-driven 92

AWS Big Data

OCTOBER 30, 2024

Ali Tore, Senior Vice President of Advanced Analytics at Salesforce, highlighting the value of this integration, says “We’re excited to partner with Amazon to bring Tableau’s powerful data exploration and AI-driven analytics capabilities to customers managing data across organizational boundaries with Amazon DataZone.

Visualization 115

Visualization Data Lake Testing Data Governance 115

CIO Business Intelligence

APRIL 8, 2025

We also examine how centralized, hybrid and decentralized data architectures support scalable, trustworthy ecosystems. As data-centric AI, automated metadata management and privacy-aware data sharing mature, the opportunity to embed data quality into the enterprises core has never been more significant.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

AWS Big Data

JANUARY 6, 2025

In this post, we show you how to establish the data ingestion pipeline between Google Analytics 4, Google Sheets, and an Amazon Redshift Serverless workgroup. With Amazon AppFlow, you can run data flows at nearly any scale and at the frequency you chooseon a schedule, in response to a business event, or on demand.

Analytics 94

Analytics Data Warehouse Big Data Metrics 94

AWS Big Data

NOVEMBER 15, 2023

The CLEA dashboards were built on the foundation of the Well-Architected Lab. For more information on this foundation, refer to A Detailed Overview of the Cost Intelligence Dashboard. Overview of the BMW Cloud Data Hub At the BMW Group, Cloud Data Hub (CDH) is the central platform for managing company-wide data and data solutions.

Dashboards 106

Dashboards Analytics Metadata Data Warehouse 106

Wayne Yaddow

MARCH 14, 2025

How dbt Core aids data teams test, validate, and monitor complex data transformations and conversions Photo by NASA on Unsplash Introduction dbt Core, an open-source framework for developing, testing, and documenting SQL-based data transformations, has become a must-have tool for modern data teams as the complexity of data pipelines grows.

Data Transformation 52

Data Transformation Testing Unstructured Data Data Quality 52

AWS Big Data

NOVEMBER 27, 2024

Within seconds of transactional data being written into Amazon Aurora (a fully managed modern relational database service offering performance and high availability at scale), the data is seamlessly made available in Amazon Redshift for analytics and machine learning. Create dbt models in dbt Cloud.

Data Warehouse 93

Data Warehouse Analytics Testing Sales 93

Sisense

OCTOBER 30, 2019

A critical part of effectively exploring your data, transforming it into actionable insights, and enhancing decision-making for your business is being empowered to slice and dice your data, and be less dependent on technical resources for new updates. What developments in report management improve reporting?

Data-driven 70

Data-driven Reporting Management Slice and Dice 70

AWS Big Data

JULY 18, 2024

OpenSearch is an open source, distributed search engine suitable for a wide array of use-cases such as ecommerce search, enterprise search (content management search, document search, knowledge management search, and so on), site search, application search, and semantic search. You use the schema API to manage schema.

Dashboards 98

Dashboards Testing Data-driven Visualization 98

FineReport

MARCH 7, 2024

In the realm of big data utilization , we often romanticize its profound impact, envisioning scenarios like precision-targeted advertising, streamlined social security management, and the intelligent evolution of the pharmaceutical sector. Try FineReport Now 1. Try FineReport Now 1.1

Dashboards 52

Dashboards Reporting Advertising Statistics 52

CIO Business Intelligence

JULY 5, 2022

What is the difference between business analytics and data analytics? Business analytics is a subset of data analytics. Data analytics is used across disciplines to find trends and solve problems using data mining , data cleansing, data transformation, data modeling, and more.

Business Analytics 136

Business Analytics Prescriptive Analytics Data mining Diagnostic Analytics 136

DataKitchen

OCTOBER 5, 2021

Data operations (or data production) is a series of pipeline procedures that take raw data, progress through a series of processing and transformation steps, and output finished products in the form of dashboards, predictions, data warehouses or whatever the business requires. Their product is the data.

Testing 152

Testing Dashboards Measurement Experimentation 152

CIO Business Intelligence

APRIL 30, 2024

This feature enables users to save calculations from a Tableau dashboard directly to Tableau’s metrics layer so they can monitor and track the information over time. This feature enables users to compare progress on a metric with a set benchmark or goal, allowing a sales manager to track their pipeline versus targets, for example.

Analytics 96

Analytics Metrics Visualization Dashboards 96

AWS Big Data

JULY 25, 2024

Comparing Amazon CloudSearch and Amazon OpenSearch Service CloudSearch is a fully managed service in the cloud that makes it straightforward to set up, manage, and scale a search solution for your website or application. We recommend that you use Amazon OpenSearch Ingestion to ingest data.

Cost-Benefit 116

Cost-Benefit Machine Learning Dashboards Management 116

IBM Big Data Hub

OCTOBER 16, 2023

AI governance refers to the practice of directing, managing and monitoring an organization’s AI activities. It includes processes that trace and document the origin of data, models and associated metadata and pipelines for audits.

Risk 70

Risk Modeling Management Metadata 70

AWS Big Data

NOVEMBER 9, 2023

It does this by helping teams handle the T in ETL (extract, transform, and load) processes. It allows users to write data transformation code, run it, and test the output, all within the framework it provides. This separation further simplifies data management and enhances the system’s overall performance.

Data Warehouse 118

Data Warehouse Testing Data Quality Reporting 118

AWS Big Data

MARCH 9, 2023

Amazon QuickSight is a fully managed, cloud-native business intelligence (BI) service that makes it easy to connect to your data, create interactive dashboards and reports, and share these with tens of thousands of users, either within QuickSight or embedded in your application or website. The QuickSight SDK v2.0

Slice and Dice 87

Slice and Dice Dashboards Analytics Interactive 87

AWS Big Data

DECEMBER 21, 2023

Solution overview The new native OpenSearch Service connector is a powerful tool that can help organizations unlock the full potential of their data. It enables you to efficiently read and write data from OpenSearch Service without needing to install or manage OpenSearch Service connector libraries. Choose Store a new secret.

Analytics 107

Analytics IT Data Lake Visualization 107

Cloudera

JULY 21, 2022

Cloudera users can securely connect Rill to a source of event stream data, such as Cloudera DataFlow , model data into Rill’s cloud-based Druid service, and share live operational dashboards within minutes via Rill’s interactive metrics dashboard or any connected BI solution. Data is made queryable in real time.

Metrics 87

Metrics Slice and Dice Data Warehouse Dashboards 87

O'Reilly on Data

JANUARY 17, 2023

Related to the previous point, a company could go from “raw data” to “it’s serving predictions on live data” in a single work day. You need to coordinate with stakeholders and product managers to suss out what kinds of models you need and how to embed them into the company’s processes.

Machine Learning 363

Machine Learning Predictive Modeling Software Modeling 363

AWS Big Data

DECEMBER 5, 2023

HR&A has used Amazon Redshift Serverless and CARTO to process survey findings more efficiently and create custom interactive dashboards to facilitate understanding of the results. The following are sample screenshots of the dashboards that show survey responses by zip code.

Measurement 93

Measurement Dashboards Data Warehouse Analytics 93

Sisense

OCTOBER 15, 2020

Dealing with this challenge requires caching the relevant context on the processing instances (state management) using techniques like sliding time windows. Another goal that teams dealing with streaming data may have is managing and optimizing a file system on object storage. Optimizing object storage. Step 4: Query.

Dashboards 98

Dashboards IoT Optimization Internet of Things 98

Cloudera

OCTOBER 11, 2021

Modak Nabu relies on a framework of “Botworks”, a series of micro-jobs to accomplish various data transformation steps from ingestion to profiling, and indexing. Cloudera Data Engineering within CDP provides : Fully managed Spark-on-Kubernetes service that hides the complexity running production DE workloads at scale.

Data Lake 95

Data Lake Cost-Benefit Data-driven Dashboards 95

AWS Big Data

JUNE 20, 2023

You can easily deliver data to supported destinations using the Amazon Kinesis Data Firehose integration with VPC flow logs. Kinesis Data Firehose is a fully managed service for delivering near-real-time streaming data to various destinations for storage and performing near-real-time analytics.

Dashboards 91

Dashboards Visualization Metrics Data Transformation 91

AWS Big Data

MAY 16, 2023

In this series of posts, we walk you through how we use Amazon QuickSight , a serverless, fully managed, business intelligence (BI) service that enables data-driven decision making at scale. Managing such a large number of devices can be challenging. QuickSight provides an ideal solution to meet these business needs.

Dashboards 85

Dashboards Statistics Data Collection Business Intelligence 85

DataKitchen

OCTOBER 25, 2022

He thinks he can sell his boss and the CEO on this idea, but his pitch won’t go over well when they still have more than six major data errors every month. DataOps Observability Starts with Data Journeys. Jason considers his dashboard idea but quickly realizes the complexity of building such a system.

Testing 130

Testing Data-driven Visualization Dashboards 130

AWS Big Data

OCTOBER 11, 2023

Due to this low complexity, the solution uses AWS serverless services to ingest the data, transform it, and make it available for analytics. The architecture uses AWS Lambda , a serverless, event-driven compute service that lets you run code without provisioning or managing servers. On the Datasets page, choose New data set.

Visualization 110

Visualization Dashboards Data-driven Gap analysis 110

AWS Big Data

MAY 3, 2023

This post is a continuation of How SOCAR built a streaming data pipeline to process IoT data for real-time analytics and control. SOCAR wanted to design and build a solution for a new Fleet Management System (FMS). The following figure shows an example of the data flow at SOCAR.

IoT 111

IoT Internet of Things Data Transformation Management 111

AWS Big Data

NOVEMBER 13, 2023

Amazon Redshift is a fully managed data warehousing service that offers both provisioned and serverless options, making it more efficient to run and scale analytics without having to manage your data warehouse. These upstream data sources constitute the data producer components.

Data Warehouse 104

Data Warehouse Analytics Data Lake Data Science 104

AWS Big Data

NOVEMBER 7, 2023

Second, organizations still need transformations like cleansing, deduplication, and combining datasets for analysis and machine learning (ML). For these, AWS Glue provides fast, scalable data transformation. They used Amazon Aurora MySQL zero-ETL integration with Amazon Redshift to achieve this.

Dashboards 80

Dashboards Data-driven Data Integration Data Lake 80

AWS Big Data

JANUARY 30, 2025

Diagram 1: Overall architecture of the solution, using AWS Step Functions, Amazon Redshift and Amazon S3 The following AWS services were used to shape our new ETL architecture: Amazon Redshift A fully managed, petabyte-scale data warehouse service in the cloud. Its also serverless, which means theres no infrastructure to manage.

Data Warehouse 52

Data Warehouse Data Architecture Machine Learning Data Transformation 52

CIO Business Intelligence

NOVEMBER 7, 2023

Given the importance of sharing information among diverse disciplines in the era of digital transformation, this concept is arguably as important as ever. The aim is to normalize, aggregate, and eventually make available to analysts across the organization data that originates in various pockets of the enterprise.

Data Warehouse 142

Data Warehouse Digital Transformation Data-driven Reporting 142

AWS Big Data

FEBRUARY 6, 2023

The tool provides a YARN log collector to connect Hadoop Resource Manager to collect YARN logs. Amazon QuickSight dashboards showcase the results from the analyzer. The script connects your local machine with the Hadoop primary node and communicates with Resource Manager.

Dashboards 83

Dashboards Optimization Data Lake Cost-Benefit 83