AWS Big Data

NOVEMBER 29, 2024

Customers often want to augment and enrich SAP source data with other non-SAP source data. Such analytic use cases can be enabled by building a data warehouse or data lake. Customers can now use the AWS Glue SAP OData connector to extract data from SAP.

Visualization 105

Visualization Data Processing Data-driven Cost-Benefit 105

AWS Big Data

OCTOBER 14, 2024

Enterprise data is brought into data lakes and data warehouses to carry out analytical, reporting, and data science use cases using AWS analytical services like Amazon Athena , Amazon Redshift , Amazon EMR , and so on. Subsequently, we’ll explore strategies for overcoming these challenges.

Metadata 105

Metadata Data Lake Modeling Data Warehouse 105

DataKitchen

APRIL 13, 2021

Read the complete blog below for a more detailed description of the vendors and their capabilities. This is not surprising given that DataOps enables enterprise data teams to generate significant business value from their data. Testing and Data Observability. Download the 2021 DataOps Vendor Landscape here.

Testing 300

Testing Machine Learning Consulting Data Science 300

Timo Elliott

JANUARY 6, 2022

Like the proverbial man looking for his keys under the streetlight , when it comes to enterprise data, if you only look at where the light is already shining, you can end up missing a lot. Remember that dark data is the data you have but don’t understand. So how do you find your dark data? Create a catalog.

IT 135

IT Metadata Data-driven Data Governance 135

AWS Big Data

DECEMBER 4, 2024

Open table formats are emerging in the rapidly evolving domain of big data management, fundamentally altering the landscape of data storage and analysis. By providing a standardized framework for data representation, open table formats break down data silos, enhance data quality, and accelerate analytics at scale.

Snapshot 84

Snapshot Metadata Data Lake Optimization 84

erwin

FEBRUARY 6, 2020

As organizations deal with managing ever more data, the need to automate data management becomes clear. Last week erwin issued its 2020 State of Data Governance and Automation (DGA) Report. One piece of the research that stuck with me is that 70% of respondents spend 10 or more hours per week on data-related activities.

Management 144

Management Data Governance Cost-Benefit Metadata 144

erwin

DECEMBER 9, 2024

When it comes to using AI and machine learning across your organization, there are many good reasons to provide your data and analytics community with an intelligent data foundation. For instance, Large Language Models (LLMs) are known to ultimately perform better when data is structured.

Metadata 52

Metadata Data Quality Sales Modeling 52

AWS Big Data

MARCH 2, 2023

Iceberg has become very popular for its support for ACID transactions in data lakes and features like schema and partition evolution, time travel, and rollback. and later supports the Apache Iceberg framework for data lakes. Apache Iceberg is designed to support these features on cost-effective petabyte-scale data lakes on Amazon S3.

Data Lake 137

Data Lake Data Processing Metadata Snapshot 137

Alation

APRIL 19, 2021

The cloud supports this new workforce, connecting remote workers to vital data, no matter their location. Data Cloud Migration Challenges and Solutions. Cloud migration is the process of moving enterprise data and infrastructure from on premise to off premise. However, cloud data migration can be difficult.

Cost-Benefit 111

Cost-Benefit Strategy Enterprise Data Strategy 111

AWS Big Data

AUGUST 19, 2024

In today’s data-driven world, the ability to seamlessly integrate and utilize diverse data sources is critical for gaining actionable insights and driving innovation. Use case Consider a large ecommerce company that relies heavily on data-driven insights to optimize its operations, marketing strategies, and customer experiences.

Analytics 119

Analytics Data-driven Data Integration Data Lake 119

AWS Big Data

MARCH 1, 2024

As they continue to implement their Digital First strategy for speed, scale and the elimination of complexity, they are always seeking ways to innovate, modernize and also streamline data access control in the Cloud. Only users with required permissions are allowed to access data in clear text.

Data Lake 119

Data Lake Data Warehouse Management Risk 119

erwin

JANUARY 30, 2020

I’m excited to share the results of our new study with Dataversity that examines how data governance attitudes and practices continue to evolve. Defining Data Governance: What Is Data Governance? . 1 reason to implement data governance. Constructing a Digital Transformation Strategy: How Data Drives Digital.

Data Governance 145

Data Governance Metadata Cost-Benefit Digital Transformation 145

erwin

APRIL 16, 2021

Data governance is best defined as the strategic, ongoing and collaborative processes involved in managing data’s access, availability, usability, quality and security in line with established internal policies and relevant data regulations. Data Governance Is Business Transformation. Predictability. Synchronicity.

Data Governance 128

Data Governance Metadata Cost-Benefit Data-driven 128

IBM Big Data Hub

MAY 20, 2024

Machine learning (ML) has become a critical component of many organizations’ digital transformation strategy. The answer lies in the data used to train these models and how that data is derived. The answer lies in the data used to train these models and how that data is derived.

Machine Learning 87

Machine Learning Modeling Metadata Strategy 87

Alation

JULY 26, 2021

How do you initiate change within a system containing many thousands of people and millions of bytes of data? During my time as a data specialist at American Family Insurance, it became clear that we had to move away from the way things had been done in the past. So you can probably imagine: The company manages a lot of data.

Insurance 98

Insurance Digital Transformation Enterprise Data Governance 98

Cloudera

NOVEMBER 13, 2020

This is part of our series of blog posts on recent enhancements to Impala. Apache Impala is synonymous with high-performance processing of extremely large datasets, but what if our data isn’t huge? It turns out that Apache Impala scales down with data just as well as it scales up. The entire collection is available here.

Optimization 143

Optimization Metadata Statistics Cost-Benefit 143

AWS Big Data

JULY 29, 2024

In the era of big data, data lakes have emerged as a cornerstone for storing vast amounts of raw data in its native format. They support structured, semi-structured, and unstructured data, offering a flexible and scalable environment for data ingestion from multiple sources.

Metadata 126

Metadata Snapshot Data Lake Metrics 126

O'Reilly on Data

NOVEMBER 28, 2023

AI users say that AI programming (66%) and data analysis (59%) are the most needed skills. Generative AI has been the biggest technology story of 2023. Almost everybody’s played with ChatGPT, Stable Diffusion, GitHub Copilot, or Midjourney. A few have even tried out Bard or Claude, or run LLaMA 1 on their laptop.

Enterprise 342

Enterprise Testing Modeling Reporting 342

AWS Big Data

OCTOBER 3, 2023

A data lake is a centralized repository that you can use to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data and then run different types of analytics for better business insights. Analytics use cases on data lakes are always evolving.

Data Lake 122

Data Lake Metadata Snapshot Recreation/Entertainment 122

AWS Big Data

SEPTEMBER 13, 2024

CFM takes a scientific approach to finance, using quantitative and systematic techniques to develop the best investment strategies. Using social network data has also often been cited as a potential source of data to improve short-term investment decisions. It was first opened to investors in 1995.

Interactive 111

Interactive Strategy Cost-Benefit Data Governance 111

erwin

MARCH 30, 2020

Data intelligence has a critical role to play in the supercomputing battle against Covid-19. While leveraging supercomputing power is a tremendous asset in our fight to combat this global pandemic, in order to deliver life-saving insights, you really have to understand what data you have and where it came from.

Metadata 122

Metadata IT Data Governance Data Quality 122

AWS Big Data

MAY 28, 2024

Apache Flink is a scalable, reliable, and efficient data processing framework that handles real-time streaming and batch workloads (but is most commonly used for real-time streaming). AWS recently announced that Apache Flink is generally available for Amazon EMR on Amazon Elastic Kubernetes Service (EKS).

Data Processing 119

Data Processing Cost-Benefit Metadata Optimization 119

erwin

MAY 31, 2019

It’s time to consider data-driven enterprise architecture. The traditional approach to enterprise architecture – the analysis, design, planning and implementation of IT capabilities for the successful execution of enterprise strategy – seems to be missing something … data. That’s right.

Data-driven 104

Data-driven Enterprise Metadata Strategy 104

erwin

JULY 22, 2020

Fostering organizational support for a data-driven culture might require a change in the organization’s culture. Recently, I co-hosted a webinar with our client E.ON , a global energy company that reinvented how it conducts business from branding to customer engagement – with data as the conduit. As an example, E.ON Avoiding Hurdles.

Data-driven 136

Data-driven Data Governance Digital Transformation Data Processing 136

AWS Big Data

MAY 24, 2023

When you build your transactional data lake using Apache Iceberg to solve your functional use cases, you need to focus on operational use cases for your S3 data lake to optimize the production environment. When the catalog property s3.delete-enabled Amazon S3 deletes expired objects on your behalf. With the s3.delete.tags

Data Lake 98

Data Lake Snapshot Metadata Optimization 98

erwin

JANUARY 14, 2021

When the pandemic first hit, there was some negative impact on big data and analytics spending. Digital transformation was accelerated, and budgets for spending on big data and analytics increased. But data without intelligence is just data, and this is WHY data intelligence is required.

Digital Transformation 143

Digital Transformation Metadata Big Data Data-driven 143

AWS Big Data

JUNE 10, 2024

Enterprises and organizations across the globe want to harness the power of data to make better decisions by putting data at the center of every decision-making process. However, throughout history, data services have held dominion over their customers’ data.

Data Lake 126

Data Lake Metadata Snapshot Analytics 126

erwin

JULY 8, 2020

As part of their transformations, businesses are moving quickly from on premise to the cloud and therefore need to create business process models available to everyone within the organization so they understand what data is tied to what applications and what processes are in place. BPM for Regulatory Compliance.

Slice and Dice 136

Slice and Dice Modeling Digital Transformation Visualization 136

AWS Big Data

FEBRUARY 21, 2024

AWS Glue is a serverless data integration service that makes it straightforward to discover, prepare, move, and integrate data from multiple sources for analytics, machine learning (ML), and application development. Furthermore, each node (driver or worker) in an AWS Glue job requires an IP address assigned from the subnet.

Strategy 116

Strategy Visualization Management IT 116

AWS Big Data

JULY 21, 2023

Data-driven organizations treat data as an asset and use it across different lines of business (LOBs) to drive timely insights and better business decisions. This leads to having data across many instances of data warehouses and data lakes using a modern data architecture in separate AWS accounts.

Data Lake 98

Data Lake Data Warehouse Marketing Management 98

Cloudera

FEBRUARY 22, 2022

Over the past decade, the successful deployment of large scale data platforms at our customers has acted as a big data flywheel driving demand to bring in even more data, apply more sophisticated analytics, and on-board many new data practitioners from business analysts to data scientists. Key Design Goals .

Snapshot 110

Snapshot Metadata Cost-Benefit Data Architecture 110

Cloudera

AUGUST 24, 2020

In the last blog with Deloitte’s Marc Beierschoder, we talked about what the hybrid cloud is, why it can benefit a business and what the key blockers often are in implementation. When building your data foundation, how can you prioritize innovation within a hybrid cloud strategy? You can read it here. .

Strategy 81

Strategy Modeling Data-driven Metadata 81

Cloudera

AUGUST 20, 2021

How CDP Enables and Accelerates Data Product Ecosystems. A multi-purpose platform focused on diverse value propositions for data products. As a result, CDP-enabled data products can meet multiple and varying functional and non-functional requirements that correspond to product attributes, each fulfilling specific customer needs.

Strategy 98

Strategy Cost-Benefit Visualization Data Warehouse 98

AWS Big Data

APRIL 24, 2023

Building a data lake on Amazon Simple Storage Service (Amazon S3) provides numerous benefits for an organization. However, many use cases, like performing change data capture (CDC) from an upstream relational database to an Amazon S3-based data lake, require handling data at a record level.

Data Lake 98

Data Lake Data Governance Machine Learning Cost-Benefit 98

erwin

SEPTEMBER 19, 2019

Data governance isn’t a one-off project with a defined endpoint. Data governance, today, comes back to the ability to understand critical enterprise data within a business context, track its physical existence and lineage, and maximize its value while ensuring quality and security. Passing the Data Governance Ball.

Data Governance 108

Data Governance Cost-Benefit Enterprise Modeling 108

AWS Big Data

MARCH 23, 2023

We recently announced support for AWS Lake Formation fine-grained access control policies in Amazon Athena queries for data stored in any supported file format using table formats such as Apache Iceberg, Apache Hudi and Apache Hive. Lake Formation provides an authorization and governance layer on data stored in Amazon S3.

Interactive 97

Interactive Snapshot Data Lake Software 97

AWS Big Data

SEPTEMBER 26, 2024

In today’s rapidly evolving digital landscape, enterprises across regulated industries face a critical challenge as they navigate their digital transformation journeys: effectively managing and governing data from legacy systems that are being phased out or replaced.

Data Governance 116

Data Governance Enterprise Management Data Lake 116