AWS Big Data

MARCH 9, 2023

Amazon QuickSight is a fully managed, cloud-native business intelligence (BI) service that makes it easy to connect to your data, create interactive dashboards and reports, and share these with tens of thousands of users, either within QuickSight or embedded in your application or website. SDK Feature overview The QuickSight SDK v2.0

Slice and Dice 94

Slice and Dice Dashboards Analytics Interactive 94

Cloudera

DECEMBER 9, 2022

Developers need to onboard new data sources, chain multiple data transformation steps together, and explore data as it travels through the flow. Interactivity when needed while saving costs. With NiFi you can configure your source processor and run it independently of any other processors to retrieve data.

Testing 100

Testing Cost-Benefit Interactive Visualization 100

AWS Big Data

OCTOBER 11, 2023

Due to this low complexity, the solution uses AWS serverless services to ingest the data, transform it, and make it available for analytics. The Data Catalog contains the table definition, which contains metadata about the data in the machine-readable file.

Visualization 116

Visualization Dashboards Data-driven Gap analysis 116

AWS Big Data

OCTOBER 18, 2023

FINRA centralizes all its data in Amazon Simple Storage Service (Amazon S3) with a remote Hive metastore on Amazon Relational Database Service (Amazon RDS) to manage their metadata information. Note: -Your query environment must have the Hive Client tool installed and a connection to your Hive metastore or AWS Glue Data Catalog.

Big Data 98

Big Data Data Processing Interactive Testing 98

Cloudera

MARCH 14, 2023

Once a draft has been created or opened, developers use the visual Designer to build their data flow logic and validate it using interactive test sessions. In the DataFlow Designer, you can create Test Sessions to turn the canvas into an interactive interface that gives you all the feedback you need to quickly iterate your flow design.

Testing 84

Testing Publishing Metadata Interactive 84

AWS Big Data

MARCH 13, 2024

You can also use the data transformation feature of Data Firehose to invoke a Lambda function to perform data transformation in batches. Athena is used to run geospatial queries on the location data stored in the S3 buckets. Choose Run. You’re now ready to query the tables using Athena.

Analytics 116

Analytics IoT Metadata Internet of Things 116

AWS Big Data

NOVEMBER 16, 2023

Data Vault 2.0 allows for the following: Agile data warehouse development Parallel data ingestion A scalable approach to handle multiple data sources even on the same entity A high level of automation Historization Full lineage support However, Data Vault 2.0

Enterprise 104

Enterprise Data Warehouse Data Lake Optimization 104

IBM Big Data Hub

OCTOBER 16, 2023

It includes processes that trace and document the origin of data, models and associated metadata and pipelines for audits. They are used in everything from robotics to tools that reason and interact with humans. Capture and document model metadata for report generation. Track models and drive transparent processes.

Risk 70

Risk Modeling Management Metadata 70

Octopai

AUGUST 11, 2024

By reverse-engineering, parsing, and converting scripts, Octopai seamlessly connects all data points within and across organizational systems. While open-source tools such as Apache Atlas, Open Metadata, Egeria, Spline, and OpenLineage offer valuable capabilities, they come with their own sets of pros and cons.

Metadata 40

Metadata Visualization Data Quality Data Governance 40

AWS Big Data

FEBRUARY 6, 2023

For data pipeline orchestration, the Apache Airflow UI is a user-friendly tool that provides detailed views into your data pipeline. When it comes to pipeline health management, each service that your tasks are interacting with could be storing or publishing logs to different locations, such as an S3 bucket or Amazon CloudWatch logs.

Management 96

Management Interactive Publishing Metadata 96

IBM Big Data Hub

JANUARY 18, 2023

This adds an additional ETL step, making the data even more stale. Data lakehouse was created to solve these problems. The data warehouse storage layer is removed from lakehouse architectures. Instead, continuous data transformation is performed within the BLOB storage. Data fabric promotes data discoverability.

Data Lake 52

Data Lake Data Warehouse Data-driven Metadata 52

IBM Big Data Hub

JULY 17, 2023

While they require task-specific labeled data for fine tuning, they also offer clients the best cost performance trade-off for non-generative use cases. offers a Prompt Lab, where users can interact with different prompts using prompt engineering on generative AI models for both zero-shot prompting and few-shot prompting.

Machine Learning 62

Machine Learning Data Warehouse Modeling Cost-Benefit 62

AWS Big Data

JANUARY 17, 2024

Incremental query refers to a query strategy that focuses on processing and analyzing only the new or updated data within a data lake since the last query. The key idea behind incremental queries is to use metadata or change tracking mechanisms to identify the new or modified data since the last query.

Data Lake 112

Data Lake Snapshot Big Data Data-driven 112

Alation

JANUARY 17, 2023

These help data analysts visualize key insights that can help you make better data-backed decisions. ELT Data Transformation Tools: ELT data transformation tools are used to extract, load, and transform your data. Examples of data transformation tools include dbt and dataform.

Data Warehouse 52

Data Warehouse Cost-Benefit Data Science Data Transformation 52

AWS Big Data

NOVEMBER 29, 2023

dbt is an open source, SQL-first templating engine that allows you to write repeatable and extensible data transforms in Python and SQL. dbt is predominantly used by data warehouses (such as Amazon Redshift ) customers who are looking to keep their data transform logic separate from storage and engine.

Data Lake 120

Data Lake Management Metrics Data Warehouse 120

AWS Big Data

JANUARY 27, 2023

With a unified data catalog, you can quickly search datasets and figure out data schema, data format, and location. The AWS Glue Data Catalog provides a uniform repository where disparate systems can store and find metadata to keep track of data in data silos. Refer to Catalogs for more information.

Data Lake 72

Data Lake Metadata Business Analysis Data-driven 72

CIO Business Intelligence

AUGUST 3, 2023

In the past, First Service Credit Union’s Chief data officer Ty Robbins struggled to integrate data from the legacy, non-relational, and often proprietary tabular databases on which many credit unions run. Start early The time to standardize everything from data modeling to its security is when the data is acquired. “We

Analytics 98

Analytics Data Lake Metadata Cost-Benefit 98

Cloudera

OCTOBER 11, 2021

The platform converges data cataloging, data ingestion, data profiling, data tagging, data discovery, and data exploration into a unified platform, driven by metadata. Modak Nabu automates repetitive tasks in the data preparation process and thus accelerates the data preparation by 4x.

Data Lake 95

Data Lake Cost-Benefit Data-driven Dashboards 95

IBM Big Data Hub

JULY 5, 2023

This was, without a question, a significant departure from traditional analytic environments, which often meant vendor-lock in and the inability to work with data at scale. Another unexpected challenge was the introduction of Spark as a processing framework for big data. Comprehensive data security and data governance (i.e.

Data Lake 60

Data Lake Metadata Cost-Benefit Data Warehouse 60

AWS Big Data

DECEMBER 21, 2023

After the data lands in Amazon S3, smava uses the AWS Glue Data Catalog and crawlers to automatically catalog the available data, capture the metadata, and provide an interface that allows querying all data assets. The data products from the Business Vault and Data Mart stages are now available for consumers.

Data Lake 109

Data Lake Data Warehouse Data-driven B2B 109

CIO Business Intelligence

APRIL 30, 2024

“But to us, it’s more than just having a data strategy; it’s also about building a great foundation of a data culture.” That’s where Tableau sees Pulse and Einstein Copilot for Tableau — a generative AI assistant that gives users the ability to interact with Tableau using natural language — coming in.

Analytics 96

Analytics Metrics Visualization Dashboards 96

AWS Big Data

JANUARY 29, 2025

Amazon EMR has long been the leading solution for processing big data in the cloud. Amazon EMR is the industry-leading big data solution for petabyte-scale data processing, interactive analytics, and machine learning using over 20 open source frameworks such as Apache Hadoop , Hive, and Apache Spark.

Big Data 75

Big Data Data Analytics Analytics Interactive 75

AWS Big Data

DECEMBER 2, 2024

AWS Glue establishes a secure connection to HubSpot using OAuth for authorization and TLS for data encryption in transit. AWS Glue also supports the ability to apply complex data transformations, enabling efficient data integration and preparation to meet your needs.

Data Lake 95

Data Lake Testing Data Integration Metadata 95

Jet Global

MAY 1, 2023

This is in contrast to traditional BI, which extracts insight from data outside of the app. As rich, data-driven user experiences are increasingly intertwined with our daily lives, end users are demanding new standards for how they interact with their business data. Yes—but basic dashboards won’t be enough.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52

Jet Global

FEBRUARY 23, 2024

This field guide to data mapping will explore how data mapping connects volumes of data for enhanced decision-making. Why Data Mapping is Important Data mapping is a critical element of any data management initiative, such as data integration, data migration, data transformation, data warehousing, or automation.

Data Warehouse 52

Data Warehouse Reporting Data Transformation Visualization 52

Jet Global

JULY 19, 2023

View mode must respect interactivity, responsive layout and limit operations with dashboard. New Interactive Legends for all Visuals simplifies report navigation for non-technical users. Context Menu for Non-Grouped Data provides further self-service user empowerment with our new context menu for ungrouped data.

Dashboards 52

Dashboards Visualization Reporting Interactive 52