AWS Big Data

NOVEMBER 14, 2024

The landscape of big data management has been transformed by the rising popularity of open table formats such as Apache Iceberg, Apache Hudi, and Linux Foundation Delta Lake. These formats, designed to address the limitations of traditional data storage systems, have become essential in modern data architectures.

Metadata 110

Metadata Data Warehouse Big Data Data Lake 110

Cloudera

JULY 29, 2022

Corporations are generating unprecedented volumes of data, especially in industries such as telecom and financial services industries (FSI). However, not all these organizations will be successful in using data to drive business value and increase profits. Is yours among the organizations hoping to cash in big with a big data solution?

Data Architecture 89

Data Architecture Data Strategy Risk Management Strategy 89

AWS Big Data

NOVEMBER 27, 2024

To populate source data: Run the following script on Query Editor to create the sample database DEMO_DB and tables inside DEMO_DB. To populate source data: Run the following script on Query Editor to create the sample database DEMO_DB and tables inside DEMO_DB. About the authors BP Yau is a Sr Partner Solutions Architect at AWS.

Data Warehouse 93

Data Warehouse Analytics Testing Sales 93

IBM Big Data Hub

JANUARY 5, 2023

Several factors determine the quality of your enterprise data like accuracy, completeness, consistency, to name a few. But there’s another factor of data quality that doesn’t get the recognition it deserves: your data architecture. How the right data architecture improves data quality.

Data Architecture 52

Data Architecture Data Quality Strategy Data Lake 52

AWS Big Data

OCTOBER 30, 2024

However, you can use the same file name as long as it’s from different auto-copy jobs: job_customerA_sales – s3://redshift-blogs/sales/customerA/2022-10-15-sales.csv job_customerB_sales – s3://redshift-blogs/sales/customerB/2022-10-15-sales.csv Do not update file contents. Do not overwrite existing files.

Data Warehouse 89

Data Warehouse Sales Data Lake Recreation/Entertainment 89

TDAN

OCTOBER 21, 2020

The term “big data” has been bandied about for a number of years now, and it has gotten to the point where it has been used so much that it is a part of IT culture. It’s hard to specifically define, yet everyone seems to have a good idea what is meant by it; big […].

Big Data 69

Big Data IT Data Architecture Data Strategy 69

AWS Big Data

DECEMBER 4, 2024

While traditional extract, transform, and load (ETL) processes have long been a staple of data integration due to its flexibility, for common use cases such as replication and ingestion, they often prove time-consuming, complex, and less adaptable to the fast-changing demands of modern data architectures.

Data Integration 97

Data Integration Data Lake Statistics Data-driven 97

AWS Big Data

DECEMBER 4, 2024

SageMaker brings together widely adopted AWS ML and analytics capabilities—virtually all of the components you need for data exploration, preparation, and integration; petabyte-scale big data processing; fast SQL analytics; model development and training; governance; and generative AI development.

Data Analytics 85

Data Analytics Analytics Data Lake Data Quality 85

TDAN

SEPTEMBER 17, 2019

Did you know that 90% of all data has been generated over the last 2 years? Big Data has been an important topic in the marketing scene for quite some time. It has been a major challenge for Chief Marketing Officers (CMOs) because it’s not easy to organize and extract useful insights from massive amounts of […].

Big Data 68

Big Data Marketing Data Architecture IT 68

Cloudera

JANUARY 12, 2018

I mentioned in an earlier blog titled, “Staffing your big data team, ” that data engineers are critical to a successful data journey. And the longer it takes to put a team in place, the likelier it is that your big data project will stall. That architecture exists to store, serve, and process data.

Big Data 54

Big Data Data Architecture Data Governance Reporting 54

Data Virtualization

OCTOBER 5, 2022

Reading Time: 3 minutes At the heart of every organization lies a data architecture, determining how data is accessed, organized, and used. For this reason, organizations must periodically revisit their data architectures, to ensure that they are aligned with current business goals.

Data Lake 52

Data Lake Data Architecture Data Integration Management 52

Cloudera

JUNE 7, 2022

Big data is cool again. As the company who taught the world the value of big data, we always knew it would be. But this is not your grandfather’s big data. It has evolved into something new – hybrid data. For Cloudera this is a back to the future moment. Fuel growth with speed and control.

IT 112

IT Data Architecture Unstructured Data Big Data 112

Cloudera

MAY 10, 2024

Iceberg, a high-performance open-source format for huge analytic tables, delivers the reliability and simplicity of SQL tables to big data while allowing for multiple engines like Spark, Flink, Trino, Presto, Hive, and Impala to work with the same tables, all at the same time.

Data Architecture 103

Data Architecture Data Governance Big Data Marketing 103

Cloudera

FEBRUARY 7, 2019

With the right technology now in place, ATB Financial is landing and curating more data than ever to bring data-driven insights to the business and its customers. Implementing a Modern Data Architecture. ATB Financial is also the first to use SAS Viya to interface between SAS tools and HDP. Check out our customer stories.

Big Data 67

Big Data Data Analytics Analytics Data Architecture 67

Octopai

APRIL 19, 2021

Her Twitter page is filled with interesting articles, webinars, reports, and current news surrounding data management. She tweets and retweets about topics such as data governance, data strategy, and data architecture. Datanami is a portal that posts about the latest news and updates when it comes to big data.

Metadata 59

Metadata Management Business Intelligence Data Governance 59

BizAcuity

APRIL 14, 2022

Big Data technology in today’s world. Did you know that the big data and business analytics market is valued at $198.08 Or that the US economy loses up to $3 trillion per year due to poor data quality? quintillion bytes of data which means an average person generates over 1.5 Big Data Ecosystem.

Big Data 52

Big Data Data Analytics Management Unstructured Data 52

Cloudera

JUNE 13, 2024

There were thousands of attendees at the event – lining up for book signings and meetings with recruiters to fill the endless job openings for developers experienced with MapReduce and managing Big Data. This was the gold rush of the 21st century, except the gold was data. We hope to see you there.

Big Data 103

Big Data Machine Learning Contextual Data Data Lake 103

Cloudera

FEBRUARY 22, 2022

Over the past decade, the successful deployment of large scale data platforms at our customers has acted as a big data flywheel driving demand to bring in even more data, apply more sophisticated analytics, and on-board many new data practitioners from business analysts to data scientists.

Snapshot 110

Snapshot Metadata Cost-Benefit Data Architecture 110

IBM Big Data Hub

AUGUST 4, 2023

Today, the way businesses use data is much more fluid; data literate employees use data across hundreds of apps, analyze data for better decision-making, and access data from numerous locations. This results in more marketable AI-driven products and greater accountability.

Data Architecture 59

Data Architecture Data Lake Machine Learning Data Governance 59

Cloudera

SEPTEMBER 24, 2021

One of the most substantial big data workloads over the past fifteen years has been in the domain of telecom network analytics. The Dawn of Telco Big Data: 2007-2012. Suddenly, it was possible to build a data model of the network and create both a historical and predictive view of its behaviour.

Analytics 96

Analytics IoT Cost-Benefit Big Data 96

Cloudera

NOVEMBER 18, 2020

The technological linchpin of its digital transformation has been its Enterprise Data Architecture & Governance platform. It hosts over 150 big data analytics sandboxes across the region with over 200 users utilizing the sandbox for data discovery.

Internet Publishing and Broadcasting 123

Internet Publishing and Broadcasting Data-driven Broadcasting Digital Transformation 123

IBM Big Data Hub

JUNE 18, 2019

In my last blog , I stressed the need for a modern data architecture (MDA) to underpin the next generation of the cognitive enterprise , fully harness data using the latest technologies, and sustain a

Data Strategy 86

Data Strategy Strategy Data Architecture Enterprise 86

Cloudera

SEPTEMBER 26, 2023

IBM and Cloudera’s common goal is to accelerate data-driven decision making for enterprise customers, working on defining and executing the best solution for each customer. You can now elevate your data potential and activate AI’s capabilities through the synergic integration between IBM watsonx and Cloudera.

Technology 95

Technology IT Data-driven Data Architecture 95

Cloudera

DECEMBER 1, 2023

Find out more about Cloudera Data Flow and CDP, the only hybrid data platform for modern data architectures with data anywhere here ( Public Sector, Government Big Data Business Intelligence and Analytics (cloudera.com).

Data Governance 109

Data Governance Data-driven Strategy Data Architecture 109

erwin

OCTOBER 24, 2019

Untapped data, if mined, represents tremendous potential for your organization. While there has been a lot of talk about big data over the years, the real hero in unlocking the value of enterprise data is metadata , or the data about the data.

Metadata 104

Metadata Management Data-driven Data Architecture 104

AWS Big Data

SEPTEMBER 7, 2023

Customers want to know how their data is being accessed, when it is being accessed, and who is accessing it. With exponential growth in data volume, centralized monitoring becomes challenging. It is also crucial to audit granular data access for security and compliance needs. Big Data Architect.

Metadata 111

Metadata Dashboards Metrics Visualization 111

Cloudera

JUNE 6, 2024

Recently, Cloudera, alongside OCBC, were named winners in the“ Best Big Data and Analytics Infrastructure Implementation ” category at The Asian Banker’s Financial Technology Innovation Awards 2024. Learn more about how Cloudera helped OCBC unlock business value with trusted data.

Contextual Data 83

Contextual Data Data Lake Data-driven Risk 83

Cloudera

JULY 27, 2021

Speaking to Aaron Boasman-Patel, Vice President, AI & Customer Experience at the TM Forum, he says the industry is all-in on Big Data. Big Data has long been a growth area in telecom,’ he told me. Reach out to me personally to learn more or follow my viewpoint through blogs and other channels at cloudera.com/blogs.

Big Data 64

Big Data Data Architecture Data Science Enterprise 64

IBM Big Data Hub

APRIL 10, 2023

A well-designed data architecture should support business intelligence and analysis, automation, and AI—all of which can help organizations to quickly seize market opportunities, build customer value, drive major efficiencies, and respond to risks such as supply chain disruptions.

Strategy 85

Strategy Data Architecture Cost-Benefit Reporting 85

AWS Big Data

MARCH 6, 2025

About the Authors Songzhi Liu is a Principal Big Data Architect with the AWS Identity Solutions team. He has over 19 years of experience architecting, building, leading, and maintaining big data platforms. Rohit Vashishtha is a Senior Analytics Specialist Solutions Architect at AWS based in Dallas, Texas.

Visualization 81

Visualization Sales Data Warehouse Management 81

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. By decoupling storage and compute, data lakes promote cost-effective storage and processing of big data. Why did Orca choose Apache Iceberg?

Data Lake 93

Data Lake Analytics Snapshot Data Quality 93

Cloudera

MARCH 13, 2019

The Data Visionary, Data Scientist, Data Architect, and HCC Community Champion awards are given out to organizations transforming their businesses through Big Data. Who are the Data Heroes? The post Introducing the 2019 Data Heroes – EMEA! appeared first on Cloudera Blog.

Data Architecture 69

Data Architecture Deep Learning Machine Learning Big Data 69

Cloudera

JANUARY 26, 2022

Modern, real-time businesses require accelerated cycles of innovation that are expensive and difficult to maintain with legacy data platforms. The hybrid cloud’s premise—two data architectures fused together—gives companies options to leverage those solutions and to address decision-making criteria, on a case-by-case basis. .

Data Processing 87

Data Processing IoT Data Warehouse Cost-Benefit 87

AWS Big Data

OCTOBER 23, 2024

This blog post is co-written with Pinar Yasar from Getir. Amazon Redshift is a fully managed cloud data warehouse that’s used by tens of thousands of customers for price-performance, scale, and advanced data analytics.

Data Warehouse 96

Data Warehouse Cost-Benefit Data Lake Data-driven 96

Cloudera

APRIL 25, 2019

A key pillar of this strategy was the “One Bank Data Architecture,” which called for a centralized data management platform. The post Breaking Down Data Silos in Financial Services with a Centralized Data Management Platform appeared first on Cloudera Blog. Explore more customer use cases.

Management 62

Management Insurance Big Data Data Architecture 62

AWS Big Data

AUGUST 31, 2023

You can download the files and then use Athena to convert the Parquet dataset into an Iceberg table, or refer to Build an Apache Iceberg data lake using Amazon Athena, Amazon EMR, and AWS Glue blog post to create the Iceberg table. In this post, we use Athena to convert the data.

Data Lake 107

Data Lake Data Warehouse Metadata Data Architecture 107

AWS Big Data

JULY 28, 2023

In this post, we show how to create and query views on federated data sources in a data mesh architecture featuring data producers and consumers. The term data mesh refers to a data architecture with decentralized data ownership. The following diagram depicts our data architecture.

Big Data 87

Big Data Data Architecture Data Lake Interactive 87