David Menninger's Analyst Perspectives

SEPTEMBER 22, 2022

Organizations are collecting data from multiple data sources and a variety of systems to enrich their analytics and business intelligence (BI). But collecting data is only half of the equation. As the data grows, it becomes challenging to find the right data at the right time.

Data Lake 278

Data Lake Business Intelligence Analytics IT 278

AWS Big Data

OCTOBER 29, 2024

This led to inefficiencies in data governance and access control. AWS Lake Formation is a service that streamlines and centralizes the data lake creation and management process. The Solution: How BMW CDH solved data duplication The CDH is a company-wide data lake built on Amazon Simple Storage Service (Amazon S3).

Data Lake 108

Data Lake Sales Metadata Machine Learning 108

AWS Big Data

OCTOBER 3, 2023

A data lake is a centralized repository that you can use to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data and then run different types of analytics for better business insights. We will use AWS Region us-east-1.

Data Lake 122

Data Lake Metadata Snapshot Recreation/Entertainment 122

AWS Big Data

MARCH 13, 2025

Unified access to your data is provided by Amazon SageMaker Lakehouse , a unified, open, and secure data lakehouse built on Apache Iceberg open standards. To overcome these hurdles, many organizations are building bespoke integrations between services, tools, and homegrown access management systems.

Analytics 112

Analytics Data Lake Data Warehouse Data-driven 112

AWS Big Data

OCTOBER 30, 2024

Amazon DataZone now launched authentication supports through the Amazon Athena JDBC driver, allowing data users to seamlessly query their subscribed data lake assets via popular business intelligence (BI) and analytics tools like Tableau, Power BI, Excel, SQL Workbench, DBeaver, and more.

Visualization 119

Visualization Data Lake Testing Data Governance 119

David Menninger's Analyst Perspectives

JANUARY 22, 2021

Organizations are dealing with exponentially increasing data that ranges broadly from customer-generated information, financial transactions, edge-generated data and even operational IT server logs. A combination of complex data lake and data warehouse capabilities are required to leverage this data.

Data Lake 269

Data Lake IT Data Warehouse Data Governance 269

AWS Big Data

JUNE 9, 2023

A modern data architecture enables companies to ingest virtually any type of data through automated pipelines into a data lake, which provides highly durable and cost-effective object storage at petabyte or exabyte scale. and Delta Lake 2.3.0. Apache Iceberg 1.2.0,

Data Lake 130

Data Lake Metadata Statistics Optimization 130

AWS Big Data

AUGUST 31, 2023

Amazon Redshift is a fast, fully managed petabyte-scale cloud data warehouse that makes it simple and cost-effective to analyze all your data using standard SQL and your existing business intelligence (BI) tools. This enables you to easily manage and maintain your tables on transactional data lakes.

Data Lake 119

Data Lake Data Warehouse Metadata Data Architecture 119

AWS Big Data

MARCH 10, 2023

Since the deluge of big data over a decade ago, many organizations have learned to build applications to process and analyze petabytes of data. Data lakes have served as a central repository to store structured and unstructured data at any scale and in various formats.

Data Lake 122

Data Lake Sales Data Warehouse Snapshot 122

CIO Business Intelligence

JUNE 9, 2022

A data management platform (DMP) is a group of tools designed to help organizations collect and manage data from a wide array of sources and to create reports that help explain what is happening in those data streams. Deploying a DMP can be a great way for companies to navigate a business world dominated by data.

Management 144

Management Advertising Data Lake Sales 144

AWS Big Data

JUNE 10, 2024

This amalgamation empowers vendors with authority over a diverse range of workloads by virtue of owning the data. This authority extends across realms such as business intelligence, data engineering, and machine learning thus limiting the tools and capabilities that can be used.

Data Lake 126

Data Lake Metadata Snapshot Analytics 126

AWS Big Data

OCTOBER 30, 2024

Amazon DataZone is a data management service that makes it faster and easier for customers to catalog, discover, share, and govern data stored across AWS, on premises, and from third-party sources. In the JDBC parameters dialog box, select Using IDC auth and copy the JDBC URL. The following screenshot shows the dialog box.

Analytics 118

Analytics Visualization Data Governance Data-driven 118

CIO Business Intelligence

MAY 24, 2022

Just after launching a focused data management platform for retail customers in March, enterprise data management vendor Informatica has now released two more industry-specific versions of its Intelligent Data Management Cloud (IDMC) — one for financial services, and the other for health and life sciences.

Finance 140

Finance Management Metadata Machine Learning 140

TDAN

SEPTEMBER 14, 2021

Data lake is a newer IT term created for a new category of data store. But just what is a data lake? According to IBM, “a data lake is a storage repository that holds an enormous amount of raw or refined data in native format until it is accessed.” That makes sense. I think the […].

Data Lake 91

Data Lake Data Warehouse IT Data Strategy 91

CIO Business Intelligence

OCTOBER 9, 2024

How will organizations wield AI to seize greater opportunities, engage employees, and drive secure access without compromising data integrity and compliance? These are vital concerns that companies must address and communicate across every level of the business. An example is Dell Technologies Enterprise Data Management.

Strategy 143

Strategy Modeling Management Data Lake 143

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. First-generation – expensive, proprietary enterprise data warehouse and business intelligence platforms maintained by a specialized team drowning in technical debt.

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

CIO Business Intelligence

OCTOBER 23, 2024

But more than anything, the data platform is putting decision-making tools in the hands of our business so people can better manage their operations. How would you categorize the change management that needed to happen to build a new enterprise data platform? We thought about change in two ways.

Enterprise 119

Enterprise Dashboards KPI Data Lake 119

AWS Big Data

JANUARY 15, 2025

Their terminal operations rely heavily on seamless data flows and the management of vast volumes of data. In this post, we show you how EUROGATE uses AWS services, including Amazon DataZone , to make data discoverable by data consumers across different business units so that they can innovate faster.

IoT 111

IoT Machine Learning Metadata Data-driven 111

CIO Business Intelligence

AUGUST 6, 2024

La firma de consultoría también deberá crear un data lake que permita almacenar y compartir fácilmente los datos de todo el ecosistema deportivo español, para lograr sinergias en torno a los distintos proyectos que se realicen.

Data Lake 118

Data Lake Software Digital Transformation Strategy 118

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

Data Lake 98

Data Lake Analytics Snapshot Data Quality 98

AWS Big Data

APRIL 27, 2023

Amazon Athena supports the MERGE command on Apache Iceberg tables, which allows you to perform inserts, updates, and deletes in your data lake at scale using familiar SQL statements that are compliant with ACID (Atomic, Consistent, Isolated, Durable). Navigate to the Athena console and choose Query editor.

Data Lake 97

Data Lake Snapshot Optimization Data Transformation 97

CIO Business Intelligence

DECEMBER 20, 2024

In addition to its flight status LLM, United has also developed LLMs for procurement and another LLM for enhancing manager-employee communications. Another new United gen AI app, currently in beta at customer sites, summarizes all relevant operational data required when there is a shift change, the CIO says.

IT 131

IT Unstructured Data Experimentation Data Lake 131

AWS Big Data

JUNE 23, 2023

Events and many other security data types are stored in Imperva’s Threat Research Multi-Region data lake. Imperva harnesses data to improve their business outcomes. As part of their solution, they are using Amazon QuickSight to unlock insights from their data.

Data Lake 95

Data Lake Dashboards Cost-Benefit Data Warehouse 95

AWS Big Data

AUGUST 1, 2023

In the world of software engineering and development, organizations use project management tools like Atlassian Jira Cloud. Managing projects with Jira leads to rich datasets, which can provide historical and predictive insights about project and development efforts. An AWS account and a login with access to the AWS Management Console.

Data Lake 98

Data Lake Data Transformation Data-driven Cost-Benefit 98

AWS Big Data

AUGUST 3, 2023

Data analytics on operational data at near-real time is becoming a common need. Due to the exponential growth of data volume, it has become common practice to replace read replicas with data lakes to have better scalability and performance. Apache Hudi connector for AWS Glue For this post, we use AWS Glue 4.0,

Data Lake 98

Data Lake Visualization Dashboards Insurance 98

CIO Business Intelligence

SEPTEMBER 22, 2023

Data management platform definition A data management platform (DMP) is a suite of tools that helps organizations to collect and manage data from a wide array of first-, second-, and third-party sources and to create reports and build customer profiles as part of targeted personalization campaigns.

Management 105

Management Advertising Data Lake Sales 105

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 98

Data Lake Data Governance Machine Learning Cost-Benefit 98

CIO Business Intelligence

FEBRUARY 27, 2025

Giving the mobile workforce access to this data via the cloud allows them to be productive from anywhere, fosters collaboration, and improves overall strategic decision-making. Connecting mainframe data to the cloud also has financial benefits as it leads to lower mainframe CPU costs by leveraging cloud computing for data transformations.

Metadata 98

Metadata Data Lake Cost-Benefit Forecasting 98

AWS Big Data

OCTOBER 30, 2024

Amazon Redshift is a fast, scalable, secure, and fully managed cloud data warehouse that makes it simple and cost-effective to analyze your data using standard SQL and your existing business intelligence (BI) tools. Raza Hafeez is a Senior Product Manager at Amazon Redshift.

Data Warehouse 110

Data Warehouse Sales Data Lake Recreation/Entertainment 110

CIO Business Intelligence

OCTOBER 2, 2024

According to Kari Briski, VP of AI models, software, and services at Nvidia, successfully implementing gen AI hinges on effective data management and evaluating how different models work together to serve a specific use case. Data management, when done poorly, results in both diminished returns and extra costs.

Management 141

Management Data Governance Cost-Benefit Structured Data 141

CIO Business Intelligence

FEBRUARY 20, 2025

Decades-old apps designed to retain a limited amount of data due to storage costs at the time are also unlikely to integrate easily with AI tools, says Brian Klingbeil, chief strategy officer at managed services provider Ensono. CIOs should also use data lakes to aggregate information from multiple sources, he adds.

Insurance 108

Insurance Cost-Benefit Unstructured Data Data Lake 108

CIO Business Intelligence

AUGUST 18, 2022

Previously, Walgreens was attempting to perform that task with its data lake but faced two significant obstacles: cost and time. Those challenges are well-known to many organizations as they have sought to obtain analytical knowledge from their vast amounts of data. Lakehouses redeem the failures of some data lakes.

Data Lake 140

Data Lake Data Warehouse Unstructured Data Business Intelligence 140

Cloudera

OCTOBER 23, 2024

In August, we wrote about how in a future where distributed data architectures are inevitable, unifying and managing operational and business metadata is critical to successfully maximizing the value of data, analytics, and AI.

Metadata 84

Metadata Data Lake Dashboards Interactive 84

AWS Big Data

JANUARY 26, 2023

AWS Glue provides an extensible architecture that enables users with different data processing use cases. A common use case is building data lakes on Amazon Simple Storage Service (Amazon S3) using AWS Glue extract, transform, and load (ETL) jobs.

Data Lake 136

Data Lake Big Data Software Interactive 136

David Menninger's Analyst Perspectives

OCTOBER 31, 2024

I previously wrote about the importance of open table formats to the evolution of data lakes into data lakehouses. The concept of the data lake was initially proposed as a single environment where data could be combined from multiple sources to be stored and processed to enable analysis by multiple users for multiple purposes.

Data Lake 130

Data Lake Unstructured Data Data Warehouse Software 130

CIO Business Intelligence

MAY 21, 2024

IBM has showcased its new generative AI -driven Concert offering that is designed to help enterprises monitor and manage their applications. Amalgam Insight’s chief analyst Hyoun Park, on the other hand, thinks that IBM Concert superficially has traits that are common with IT asset management offerings.

Enterprise 110

Enterprise Management Data Lake Risk 110

AWS Big Data

NOVEMBER 29, 2023

We have seen a strong customer demand to expand its scope to cloud-based data lakes because data lakes are increasingly the enterprise solution for large-scale data initiatives due to their power and capabilities. The team uses dbt-glue to build a transformed gold model optimized for business intelligence (BI).

Data Lake 122

Data Lake Management Metrics Data Warehouse 122