AWS Big Data

MARCH 10, 2023

Since the deluge of big data over a decade ago, many organizations have learned to build applications to process and analyze petabytes of data. Data lakes have served as a central repository to store structured and unstructured data at any scale and in various formats.

Data Lake 122

Data Lake Sales Data Warehouse Snapshot 122

AWS Big Data

JUNE 9, 2023

A modern data architecture enables companies to ingest virtually any type of data through automated pipelines into a data lake, which provides highly durable and cost-effective object storage at petabyte or exabyte scale.

Data Lake 130

Data Lake Metadata Statistics Optimization 130

Smart Data Collective

AUGUST 13, 2019

Machine learning is rewriting the rules of the gaming industry. One report showed that Caesars is investing $1 billion in big data. I still remember playing my favorite games growing up, before machine learning was a thing or big data was a household word. Other companies are following suit.

Machine Learning 96

Machine Learning Big Data Data Lake Strategy 96

AWS Big Data

SEPTEMBER 10, 2024

Organizations are increasingly using a multi-cloud strategy to run their production workloads. We often see requests from customers who have started their data journey by building data lakes on Microsoft Azure, to extend access to the data to AWS services. For this post, we use the Shared Key authentication method.

Data Lake 111

Data Lake Metadata Management Software 111

Jet Global

SEPTEMBER 4, 2020

There is an established body of practice around creating, managing, and accessing OLAP data (known as “cubes”). Data Lakes. There has been a lot of talk over the past year or two in the D365F&SCM world about “data lakes.” Traditional databases and data warehouses do not lend themselves to that task.

Data Lake 88

Data Lake OLAP Data Warehouse Unstructured Data 88

Jet Global

NOVEMBER 5, 2020

Option 3: Azure Data Lakes. This leads us to Microsoft’s apparent long-term strategy for D365 F&SCM reporting: Azure Data Lakes. Azure Data Lakes are highly complex and designed with a different fundamental purpose in mind than financial and operational reporting. Azure Data Lakes are complicated.

Data Lake 98

Data Lake OLAP Data Warehouse Unstructured Data 98

AWS Big Data

JANUARY 9, 2025

In our previous post Backtesting index rebalancing arbitrage with Amazon EMR and Apache Iceberg , we showed how to use Apache Iceberg in the context of strategy backtesting. Our analysis shows that Iceberg can accelerate query performance by up to 52%, reduce operational costs, and significantly improve data management at scale.

Metadata 111

Metadata Snapshot Cost-Benefit Optimization 111

CIO Business Intelligence

FEBRUARY 20, 2025

Decades-old apps designed to retain a limited amount of data due to storage costs at the time are also unlikely to integrate easily with AI tools, says Brian Klingbeil, chief strategy officer at managed services provider Ensono. The aim is to create integration pipelines that seamlessly connect different systems and data sources.

Insurance 108

Insurance Cost-Benefit Unstructured Data Data Lake 108

AWS Big Data

SEPTEMBER 13, 2023

A modern data architecture is an evolutionary architecture pattern designed to integrate a data lake, data warehouse, and purpose-built stores with a unified governance model. The company wanted the ability to continue processing operational data in the secondary Region in the rare event of primary Region failure.

Data Lake 111

Data Lake Data Processing Metadata Snapshot 111

Sisense

DECEMBER 23, 2019

Artificial Intelligence and machine learning are the future of every industry, especially data and analytics. Let’s talk about AI and machine learning (ML). AI and ML are the only ways to derive value from massive data lakes, cloud-native data warehouses, and other huge stores of information.

Data Lake 86

Data Lake Machine Learning Data Warehouse Digital Transformation 86

CIO Business Intelligence

DECEMBER 20, 2024

Beyond breaking down silos, modern data architectures need to provide interfaces that make it easy for users to consume data using tools fit for their jobs. Data must be able to freely move to and from data warehouses, data lakes, and data marts, and interfaces must make it easy for users to consume that data.

Data Architecture 122

Data Architecture Management Consulting Internet of Things 122

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. In this post, we describe Orca’s journey building a transactional data lake using Amazon Simple Storage Service (Amazon S3), Apache Iceberg, and AWS Analytics.

Data Lake 98

Data Lake Analytics Snapshot Data Quality 98

David Menninger's Analyst Perspectives

JANUARY 21, 2025

It continues to position its document database product as a developer data platform which is primarily used to support the development and deployment of net-new applications rather than as a direct replacement for relational databases. The recent launch of MongoDB 8.0

Data Lake 130

Data Lake IoT Cost-Benefit Enterprise 130

AWS Big Data

JUNE 23, 2023

Events and many other security data types are stored in Imperva’s Threat Research Multi-Region data lake. Imperva harnesses data to improve their business outcomes. As part of their solution, they are using Amazon QuickSight to unlock insights from their data.

Data Lake 97

Data Lake Dashboards Cost-Benefit Data Warehouse 97

Corinium

JUNE 6, 2019

Some of the work is very foundational, such as building an enterprise data lake and migrating it to the cloud, which enables other more direct value-added activities such as self-service. Newer methods can work with large amounts of data and are able to unearth latent interactions.

Insurance 250

Insurance Analytics Forecasting Deep Learning 250

CIO Business Intelligence

APRIL 8, 2025

I aim to outline pragmatic strategies to elevate data quality into an enterprise-wide capability. However, even the most sophisticated models and platforms can be undone by a single point of failure: poor data quality. This challenge remains deceptively overlooked despite its profound impact on strategy and execution.

Data Quality 102

Data Quality Data-driven Key Performance Indicator Metadata 102

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 98

Data Lake Data Governance Machine Learning Cost-Benefit 98

CIO Business Intelligence

AUGUST 9, 2024

At Atlanta’s Hartsfield-Jackson International Airport, an IT pilot has led to a wholesale data journey destined to transform operations at the world’s busiest airport, fueled by machine learning and generative AI. Data integrity presented a major challenge for the team, as there were many instances of duplicate data.

Data Transformation 126

Data Transformation Machine Learning Data Lake Dashboards 126

CIO Business Intelligence

APRIL 12, 2023

Real-time AI involves processing data for making decisions within a given time frame. Real-time AI brings together streaming data and machine learning algorithms to make fast and automated decisions; examples include recommendations, fraud detection, security monitoring, and chatbots. It isn’t easy.

Machine Learning 145

Machine Learning Cost-Benefit Data-driven Strategy 145

CIO Business Intelligence

MARCH 29, 2024

That’s why Rocket Mortgage has been a vigorous implementor of machine learning and AI technologies — and why CIO Brian Woodring emphasizes a “human in the loop” AI strategy that will not be pinned down to any one generative AI model. It’s a powerful strategy.” The rest are on premises.

Data Lake 134

Data Lake Machine Learning Data Warehouse Unstructured Data 134

Cloudera

DECEMBER 14, 2022

The company recently migrated to Cloudera Data Platform (CDP ) and CDP Machine Learning to power a number of solutions that have increased operational efficiency, enabled new revenue streams and improved risk management. OCBC also won a Cloudera Data Impact Award 2022 in the Transformation category for the project.

Data Strategy 81

Data Strategy Strategy IT Contextual Data 81

CIO Business Intelligence

MARCH 9, 2022

However, they do contain effective data management, organization, and integrity capabilities. As a result, users can easily find what they need, and organizations avoid the operational and cost burdens of storing unneeded or duplicate data copies. Warehouse, data lake convergence. Meet the data lakehouse.

Data Lake 119

Data Lake Unstructured Data Data Warehouse Big Data 119

CIO Business Intelligence

AUGUST 18, 2022

Previously, Walgreens was attempting to perform that task with its data lake but faced two significant obstacles: cost and time. Those challenges are well-known to many organizations as they have sought to obtain analytical knowledge from their vast amounts of data. Lakehouses redeem the failures of some data lakes.

Data Lake 140

Data Lake Data Warehouse Unstructured Data Business Intelligence 140

AWS Big Data

JULY 29, 2024

In the era of big data, data lakes have emerged as a cornerstone for storing vast amounts of raw data in its native format. They support structured, semi-structured, and unstructured data, offering a flexible and scalable environment for data ingestion from multiple sources.

Metadata 126

Metadata Snapshot Data Lake Metrics 126

AWS Big Data

OCTOBER 14, 2024

Enterprise data is brought into data lakes and data warehouses to carry out analytical, reporting, and data science use cases using AWS analytical services like Amazon Athena , Amazon Redshift , Amazon EMR , and so on. Subsequently, we’ll explore strategies for overcoming these challenges.

Metadata 105

Metadata Data Lake Modeling Data Warehouse 105

AWS Big Data

OCTOBER 21, 2024

However, enterprises often encounter challenges with data silos, insufficient access controls, poor governance, and quality issues. Embracing data as a product is the key to address these challenges and foster a data-driven culture. To achieve this, they plan to use machine learning (ML) models to extract insights from data.

Sales 116

Sales Data-driven Data Processing Key Performance Indicator 116

CIO Business Intelligence

JUNE 29, 2022

Taking the broadest possible interpretation of data analytics , Azure offers more than a dozen services — and that’s before you include Power BI, with its AI-powered analysis and new datamart option , or governance-oriented approaches such as Microsoft Purview. Azure Data Lake Analytics. Microsoft. Azure Analysis Services.

Data Lake 116

Data Lake Analytics Data Warehouse Machine Learning 116

AWS Big Data

NOVEMBER 29, 2023

This cloud service was a significant leap from the traditional data warehousing solutions, which were expensive, not elastic, and required significant expertise to tune and operate. Use one click to access your data lake tables using auto-mounted AWS Glue data catalogs on Amazon Redshift for a simplified experience.

Data Warehouse 130

Data Warehouse Analytics Data Lake Machine Learning 130

AWS Big Data

DECEMBER 4, 2024

This post explores how the shift to a data product mindset is being implemented, the challenges faced, and the early wins that are shaping the future of data management in the Institutional Division. Amazon DataZone plays an essential role in facilitating data product management for the domain teams.

Metadata 105

Metadata Data Governance Data Quality Data-driven 105

Smart Data Collective

DECEMBER 13, 2022

It is important to note that data analytics relies on computer tools and software to collect and analyze data so that business choices may be made properly. Data analytics is widely used in business since it allows organizations to better understand their consumers and improve their advertising strategies.

Visualization 145

Visualization Key Performance Indicator Sales Advertising 145

CIO Business Intelligence

NOVEMBER 25, 2022

As part of that transformation, Agusti has plans to integrate a data lake into the company’s data architecture and expects two AI proofs of concept (POCs) to be ready to move into production within the quarter. Like many CIOs, Carhartt’s top digital leader is aware that data is the key to making advanced technologies work.

Data Lake 128

Data Lake Data Warehouse Unstructured Data Data Architecture 128

AWS Big Data

MARCH 26, 2024

A Gartner Marketing survey found only 14% of organizations have successfully implemented a C360 solution, due to lack of consensus on what a 360-degree view means, challenges with data quality, and lack of cross-functional governance structure for customer data.

Data Strategy 103

Data Strategy Strategy Data Warehouse Prescriptive Analytics 103

AWS Big Data

NOVEMBER 16, 2023

One of the core features of AWS Lake Formation is the delegation of permissions on a subset of resources such as databases, tables, and columns in AWS Glue Data Catalog to data stewards, empowering them make decisions regarding who should get access to their resources and helping you decentralize the permissions management of your data lakes.

Management 122

Management Data Lake Sales Machine Learning 122

AWS Big Data

MARCH 7, 2024

At the same time, they need to optimize operational costs to unlock the value of this data for timely insights and do so with a consistent performance. With this massive data growth, data proliferation across your data stores, data warehouse, and data lakes can become equally challenging.

Data Lake 131

Data Lake Analytics Dashboards Metrics 131

AWS Big Data

SEPTEMBER 12, 2024

This integration empowers organizations to break down data silos, accelerate analytics, and drive more agile customer-centric strategies. In his current role at Salesforce, Sriram works on Zero Copy integration with major data lake partners and helps customers deliver value with their data strategies.

Data Lake 115

Data Lake Analytics Data-driven Data Strategy 115

CIO Business Intelligence

OCTOBER 21, 2022

Chipotle IT’s secret sauce Garner credits Chipotle’s wholly owned business model for enabling him to deploy advanced technologies such as the cloud, analytics, data lake, and AI uniformly to all restaurants because they are all based on the same digital backbone. Chipotle’s digital business in 2022 was $3.5

Digital Transformation 145

Digital Transformation Data Lake Forecasting Technology 145

AWS Big Data

MARCH 4, 2024

As enterprises collect increasing amounts of data from various sources, the structure and organization of that data often need to change over time to meet evolving analytical needs. Schema evolution enables adding, deleting, renaming, or modifying columns without needing to rewrite existing data.

Snapshot 132

Snapshot Data Lake Metadata Recreation/Entertainment 132