Alation

JUNE 17, 2021

Leading companies like Cisco, Nielsen, and Finnair turn to Alation + Snowflake for data governance and analytics. By joining forces, we can build more potent, tailored solutions that leverage data governance as a competitive asset. Lastly, active data governance simplifies stewardship tasks of all kinds.

Data Governance 98

Data Governance Data Lake Insurance Enterprise 98

AWS Big Data

OCTOBER 10, 2023

Data governance is the process of ensuring the integrity, availability, usability, and security of an organization’s data. Due to the volume, velocity, and variety of data being ingested in data lakes, it can get challenging to develop and maintain policies and procedures to ensure data governance at scale for your data lake.

Data Quality 103

Data Quality Data Governance Data Lake Testing 103

AWS Big Data

SEPTEMBER 26, 2024

In this blog post, there are three personas: Data Lake Administrator (with admin level access) User Silver from the Data Engineering group User Lead Auditor from the Auditor group. You will see how different personas in an organization can access the data without the need to modify their existing enterprise entitlements.

Data Governance 104

Data Governance Enterprise Management Data Lake 104

AWS Big Data

JULY 21, 2023

Data-driven organizations treat data as an asset and use it across different lines of business (LOBs) to drive timely insights and better business decisions. This leads to having data across many instances of data warehouses and data lakes using a modern data architecture in separate AWS accounts.

Data Lake 92

Data Lake Data Warehouse Marketing Management 92

David Menninger's Analyst Perspectives

OCTOBER 7, 2021

In this post, I don’t want to debate the meanings and origins of different terms; rather, I’d like to highlight a technology weapon that you should have in your data management arsenal. We currently refer to this technology as data virtualization.

Technology 246

Technology Management Data Lake IT 246

AWS Big Data

JANUARY 31, 2023

Data governance is the collection of policies, processes, and systems that organizations use to ensure the quality and appropriate handling of their data throughout its lifecycle for the purpose of generating business value.

Data Lake 89

Data Lake Data Governance Data Architecture Machine Learning 89

David Menninger's Analyst Perspectives

OCTOBER 24, 2024

Although the terms data fabric and data mesh are often used interchangeably, I previously explained that they are distinct but complementary. Denodo remains a specialist data management software provider and in September 2023 announced that it had received a $336 million investment from asset management firm TPG.

Management 130

Management Data-driven Data Governance Data Lake 130

AWS Big Data

JANUARY 18, 2024

AWS Lake Formation and the AWS Glue Data Catalog form an integral part of a data governance solution for data lakes built on Amazon Simple Storage Service (Amazon S3) with multiple AWS analytics services integrating with them. To learn more about DataZone, refer to the User Guide. Crawlers, salut!

Data Lake 104

Data Lake Metadata Data Governance Statistics 104

AWS Big Data

JUNE 26, 2024

New feature: Custom AWS service blueprints Previously, Amazon DataZone provided default blueprints that created AWS resources required for data lake, data warehouse, and machine learning use cases. If you’re new to Amazon DataZone, refer to Getting started.

Data Lake 102

Data Lake Data Warehouse Unstructured Data Data Governance 102

Alation

FEBRUARY 20, 2020

For many enterprises, a hybrid cloud data lake is no longer a trend, but becoming reality. With an on-premise deployment, enterprises have full control over data security, data access, and data governance. Data that needs to be tightly controlled (e.g. The Problem with Hybrid Cloud Environments.

Data Lake 52

Data Lake ROI Metadata Cost-Benefit 52

AWS Big Data

JUNE 25, 2024

This post is co-authored by Vijay Gopalakrishnan, Director of Product, Salesforce Data Cloud. In today’s data-driven business landscape, organizations collect a wealth of data across various touch points and unify it in a central data warehouse or a data lake to deliver business insights.

Data Lake 111

Data Lake Cost-Benefit Data-driven Data Warehouse 111

AWS Big Data

OCTOBER 21, 2024

However, enterprises often encounter challenges with data silos, insufficient access controls, poor governance, and quality issues. Embracing data as a product is the key to address these challenges and foster a data-driven culture.

Sales 104

Sales Data-driven Data Processing Key Performance Indicator 104

AWS Big Data

FEBRUARY 13, 2025

Refer to the appendix at the end of this post for more details. To organize the data assets within the organization, the admin logs in to the SageMaker Unified Studio URL and creates domain units aligned with the business divisions. Refer to the appendix at the end of this post for more details.

Data Analytics 72

Data Analytics Analytics Modeling Management 72

CIO Business Intelligence

OCTOBER 13, 2023

Solutions data architect: These individuals design and implement data solutions for specific business needs, including data warehouses, data marts, and data lakes. Application data architect: The application data architect designs and implements data models for specific software applications.

Data Architecture 132

Data Architecture Data Warehouse Statistics Visualization 132

erwin

JULY 17, 2019

The solution is data intelligence. It improves IT and business data literacy and knowledge, supporting enterprise data governance and business enablement. Organizations need a real-time, accurate picture of the metadata landscape to: Discover data – Identify and interrogate metadata from various data management silos.

Digital Transformation 98

Digital Transformation Strategy Metadata Data-driven 98

DataKitchen

JUNE 12, 2023

I spent much time de-categorizing DataOps: we are not discussing ETL, Data Lake, or Data Science. Today we have had over 20,000 signatures , millions of page views, and copycat clones, and it is frequently used as a reference guide. It’s Customer Journey for data analytic systems.

Testing 130

Testing Dashboards Data Lake Data Science 130

AWS Big Data

OCTOBER 20, 2023

Today, we are pleased to announce new AWS Glue connectors for Azure Blob Storage and Azure Data Lake Storage that allow you to move data bi-directionally between Azure Blob Storage, Azure Data Lake Storage, and Amazon Simple Storage Service (Amazon S3). option("header","true").load("wasbs://yourblob@youraccountname.blob.core.windows.net/loadingtest-input/100mb")

Data Lake 96

Data Lake Big Data Data Warehouse Consulting 96

AWS Big Data

MARCH 27, 2023

Data lakes have come a long way, and there’s been tremendous innovation in this space. Today’s modern data lakes are cloud native, work with multiple data types, and make this data easily available to diverse stakeholders across the business. In the navigation pane, under Data catalog , choose Settings.

Data Lake 91

Data Lake Data-driven Management Data Architecture 91

AWS Big Data

MARCH 7, 2023

Flexible and easy to use – The solutions should provide less restrictive, easy-to-access, and ready-to-use data. A data hub is a center of data exchange that constitutes a hub of data repositories and is supported by data engineering, data governance, security, and monitoring services.

Analytics 78

Analytics Data Warehouse Data Lake Metadata 78

Alation

JANUARY 26, 2023

These data requirements could be satisfied with a strong data governance strategy. Governance can — and should — be the responsibility of every data user, though how that’s achieved will depend on the role within the organization. How can data engineers address these challenges directly?

Data Governance 52

Data Governance Strategy Data Quality Data Collection 52

David Menninger's Analyst Perspectives

NOVEMBER 21, 2018

Once again I attended Tableau's Users Conference, along with 17,000 other attendees, affectionately self-referred to as "data nerds".

Data Lake 160

Data Lake Data Integration Data Governance Software 160

CIO Business Intelligence

MARCH 22, 2022

Which type(s) of storage consolidation you use depends on the data you generate and collect. . One option is a data lake—on-premises or in the cloud—that stores unprocessed data in any type of format, structured or unstructured, and can be queried in aggregate. Set up unified data governance rules and processes.

Analytics 115

Analytics Key Performance Indicator Data Warehouse Data-driven 115

AWS Big Data

APRIL 28, 2023

The first post of this series describes the overall architecture and how Novo Nordisk built a decentralized data mesh architecture, including Amazon Athena as the data query engine. The third post will show how end-users can consume data from their tool of choice, without compromising data governance.

Data Governance 72

Data Governance Management Data-driven Analytics 72

AWS Big Data

MARCH 27, 2024

The solution uses AWS services such as AWS HealthLake , Amazon Redshift , Amazon Kinesis Data Streams , and AWS Lake Formation to build a 360 view of patients. Simultaneously and as part of AWS HealthLake managed service, the nested JSON FHIR data undergoes an ETL process and is stored in Apache Iceberg open table format in Amazon S3.

Data Analytics 105

Data Analytics Analytics Data Warehouse Data Lake 105

AWS Big Data

AUGUST 16, 2023

Iceberg has become very popular for its support for ACID transactions in data lakes and features like schema and partition evolution, time travel, and rollback. For more details, refer to Creating Apache Iceberg tables. If this is the first time accessing the Lake Formation console, add yourself as the data lake administrator.

Data Lake 85

Data Lake Metadata Snapshot Management 85

AWS Big Data

MAY 7, 2024

Data governance is a key enabler for teams adopting a data-driven culture and operational model to drive innovation with data. Amazon DataZone allows you to simply and securely govern end-to-end data assets stored in your Amazon Redshift data warehouses or data lakes cataloged with the AWS Glue data catalog.

Metadata 86

Metadata Data Lake Data Processing Data-driven 86

AWS Big Data

JULY 28, 2023

Amazon Redshift now makes it easier for you to run queries in AWS data lakes by automatically mounting the AWS Glue Data Catalog. You no longer have to create an external schema in Amazon Redshift to use the data lake tables cataloged in the Data Catalog.

Data Lake 72

Data Lake Data Governance Data Warehouse Modeling 72

Alation

APRIL 6, 2023

Data governance is traditionally applied to structured data assets that are most often found in databases and information systems. This blog focuses on governing spreadsheets that contain data, information, and metadata, and must themselves be governed.

Data Governance 52

Data Governance Metadata Cost-Benefit Structured Data 52

AWS Big Data

MARCH 5, 2025

In this solution (as shown in the preceding figure), the AWS account that contains the data assets is referred to as the producer account. The AWS account that needs to access or use the data from the producer account is referred to as the consumer account. You will then publish the data assets from these data sources.

Analytics 59

Analytics Publishing Metadata Sales 59

erwin

AUGUST 31, 2023

Agile Modeling : Agile methodologies introduced flexibility, allowing data models to evolve alongside changing business needs. Data Governance : Incorporating data governance principles enabled better data management, ensuring data accuracy, consistency, and compliance.

Data-driven 98

Data-driven Modeling Enterprise Structured Data 98

IBM Big Data Hub

JANUARY 17, 2023

A data lakehouse is an emerging data management architecture that improves efficiency and converges data warehouse and data lake capabilities driven by a need to improve efficiency and obtain critical insights faster. Let’s start with why data lakehouses are becoming increasingly important.

Data Lake 52

Data Lake Metadata Data Warehouse Data Governance 52

AWS Big Data

AUGUST 19, 2024

Refer to Enabling AWS PrivateLink in the Snowflake documentation to verify the steps, required access level, and service level to set the configurations. For Data sources , search for and select Snowflake. To obtain the Snowflake PrivateLink account URL, refer to parameters obtained in the prerequisites. Choose Next.

Analytics 107

Analytics Data-driven Data Integration Data Lake 107

AWS Big Data

JULY 8, 2024

Introduction to OpenLineage compatible data lineage The need to capture data lineage consistently across various analytical services and combine them into a unified object model is key in uncovering insights from the lineage artifact. To learn more, refer to Creating inventory and published data in Amazon DataZone.

Visualization 108

Visualization Metadata Publishing Sales 108

AWS Big Data

JANUARY 24, 2023

AWS Lake Formation helps with enterprise data governance and is important for a data mesh architecture. It works with the AWS Glue Data Catalog to enforce data access and governance. This solution only replicates metadata in the Data Catalog, not the actual underlying data.

Data Architecture 72

Data Architecture Metadata Data Lake Snapshot 72

Cloudera

OCTOBER 7, 2021

Those decentralization efforts appeared under different monikers through time, e.g., data marts versus data warehousing implementations (a popular architectural debate in the era of structured data) then enterprise-wide data lakes versus smaller, typically BU-Specific, “data ponds”.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

AWS Big Data

JANUARY 30, 2023

Many customers need an ACID transaction (atomic, consistent, isolated, durable) data lake that can log change data capture (CDC) from operational data sources. There is also demand for merging real-time data into batch data. Delta Lake framework provides these two capabilities.

Insurance 86

Insurance Data Lake Data-driven Management 86

IBM Big Data Hub

AUGUST 4, 2023

In turn, they both must also have the data literacy skills to be able to verify the data’s accuracy, ensure its security, and provide or follow guidance on when and how it should be used. Then, it applies these insights to automate and orchestrate the data lifecycle.

Data Architecture 59

Data Architecture Data Lake Machine Learning Data Governance 59