AWS Big Data

NOVEMBER 20, 2024

We’re excited to announce a new feature in Amazon DataZone that offers enhanced metadata governance for your subscription approval process. With this update, domain owners can define and enforce metadata requirements for data consumers when they request access to data assets.

Metadata 76

Metadata Data Governance Metrics Marketing 76

erwin

JULY 12, 2019

In light of recent, high-profile data breaches, it’s past-time we re-examined strategic data governance and its role in managing regulatory requirements. for alleged violations of the European Union’s General Data Protection Regulation (GDPR). Five Steps to GDPR/CCPA Compliance. How erwin Can Help.

Data Governance 110

Data Governance Management Metadata Risk Management 110

Octopai

NOVEMBER 10, 2021

Metadata has been defined as the who, what, where, when, why, and how of data. Without the context given by metadata, data is just a bunch of numbers and letters. But going on a rampage to define, categorize, and otherwise metadata-ize your data doesn’t necessarily give you the key to the value in your data.

Metadata 98

Metadata Management Interactive Strategy 98

AWS Big Data

AUGUST 13, 2024

Amazon DataZone has announced a set of new data governance capabilities—domain units and authorization policies—that enable you to create business unit-level or team-level organization and manage policies according to your business needs. Data domains form a foundational pillar in data governance frameworks.

Data Governance 110

Data Governance Metadata Enterprise Sales 110

Cloudera

MARCH 3, 2022

Whether the enterprise uses dozens or hundreds of data sources for multi-function analytics, all organizations can run into data governance issues. Bad data governance practices lead to data breaches, lawsuits, and regulatory fines — and no enterprise is immune. . Everyone Fails Data Governance.

Data Governance 100

Data Governance Enterprise Data Quality Metadata 100

AWS Big Data

OCTOBER 29, 2024

To achieve this, they aimed to break down data silos and centralize data from various business units and countries into the BMW Cloud Data Hub (CDH). However, the initial version of CDH supported only coarse-grained access control to entire data assets, and hence it was not possible to scope access to data asset subsets.

Data Lake 89

Data Lake Sales Metadata Machine Learning 89

AWS Big Data

OCTOBER 30, 2024

After connecting, you can query, visualize, and share data—governed by Amazon DataZone—within the tools you already know and trust. To achieve this, you need access to sales orders, shipment details, and customer data owned by the retail team. The data producer from the retail team will review and approve your subscription.

Visualization 116

Visualization Data Lake Testing Data Governance 116

AWS Big Data

OCTOBER 21, 2024

Business analysts enhance the data with business metadata/glossaries and publish the same as data assets or data products. The data security officer sets permissions in Amazon DataZone to allow users to access the data portal. Amazon Athena is used to query, and explore the data.

Sales 105

Sales Data-driven Data Processing Key Performance Indicator 105

erwin

JANUARY 24, 2019

With an automation framework, data professionals can meet these needs at a fraction of the cost of the traditional manual way. In data governance terms, an automation framework refers to a metadata-driven universal code generator that works hand in hand with enterprise data mapping for: Pre-ETL enterprise data mapping.

Data Governance 71

Data Governance Metadata Data-driven Cost-Benefit 71

erwin

MARCH 30, 2020

To do this, the consortium will need the ability to automatically scan and catalog the data sources and apply strict data governance and quality practices. Unraveling Data Complexities with Metadata Management. Metadata management will be critical to the process for cataloging data via automated scans.

Metadata 122

Metadata IT Data Governance Data Quality 122

O'Reilly on Data

APRIL 2, 2019

A few years ago, we started publishing articles (see “Related resources” at the end of this post) on the challenges facing data teams as they start taking on more machine learning (ML) projects. Metadata and artifacts needed for audits: as an example, the output from the components of MLflow will be very pertinent for audits.

Machine Learning 269

Machine Learning Modeling Data Science Software 269

erwin

JUNE 26, 2020

It’s no surprise that most organizations’ data is often fragmented and siloed across numerous sources (e.g., legacy systems, data warehouses, flat files stored on individual desktops and laptops, and modern, cloud-based repositories.). This also diminishes the value of data as an asset. Technical Metadata.

Metadata 132

Metadata Cost-Benefit Measurement Data-driven 132

erwin

JUNE 22, 2020

Earlier this year, erwin released its 2020 State of Data Governance and Automation (DGA) report. About 70 percent of the DGA report respondents – a combination of roles from data architects to executive managers – say they spend an average of 10 or more hours per week on data-related activities.

Metadata 105

Metadata Data Governance Digital Transformation Measurement 105

AWS Big Data

MARCH 5, 2025

Solution overview This solution provides a streamlined way to enable cross-account data collaboration using Amazon DataZone domain association while maintaining security and governance. Data publishers : Users in producer AWS accounts. Data subscribers : Users in consumer AWS accounts.

Analytics 59

Analytics Publishing Metadata Sales 59

David Menninger's Analyst Perspectives

OCTOBER 8, 2024

I assert that through 2027, three-quarters of enterprises will be engaged in data intelligence initiatives to increase trust in their data by leveraging metadata to understand how, when and where data is used in their organization, and by whom. Regards, Matt Aslett

Data Quality 130

Data Quality Data Governance Enterprise Visualization 130

erwin

JULY 2, 2020

This matters because, as he said, “By placing the data and the metadata into a model, which is what the tool does, you gain the abilities for linkages between different objects in the model, linkages that you cannot get on paper or with Visio or PowerPoint.” Data Governance with erwin Data Intelligence.

Enterprise 122

Enterprise Modeling Metadata Data Governance 122

AWS Big Data

OCTOBER 4, 2023

Collaboration – Analysts, data scientists, and data engineers often own different steps within the end-to-end analytics journey but do not have an simple way to collaborate on the same governed data, using the tools of their choice.

Metadata 114

Metadata Data Lake Publishing Data Governance 114

Octopai

APRIL 19, 2021

Aptly named, metadata management is the process in which BI and Analytics teams manage metadata, which is the data that describes other data. In other words, data is the context and metadata is the content. Without metadata, BI teams are unable to understand the data’s full story.

Metadata 59

Metadata Management Business Intelligence Data Governance 59

Octopai

JANUARY 19, 2020

BCBS 239 is a document published by that committee entitled, Principles for Effective Risk Data Aggregation and Risk Reporting. The document, first published in 2013, outlines best practices for global and domestic banks to identify, manage, and report risks, including credit, market, liquidity, and operational risks.

Metadata 89

Metadata Risk Management Business Intelligence Data Governance 89

AWS Big Data

JULY 8, 2024

Introduction to OpenLineage compatible data lineage The need to capture data lineage consistently across various analytical services and combine them into a unified object model is key in uncovering insights from the lineage artifact. To learn more, refer to Creating inventory and published data in Amazon DataZone.

Visualization 109

Visualization Metadata Publishing Sales 109

erwin

MAY 27, 2020

What, then, should users look for in a data modeling product to support their governance/intelligence requirements in the data-driven enterprise? Nine Steps to Data Modeling. Provide metadata and schema visualization regardless of where data is stored.

Modeling 98

Modeling Metadata Data Governance Visualization 98

IBM Big Data Hub

JULY 28, 2022

Metadata enrichment is about scaling the onboarding of new data into a governed data landscape by taking data and applying the appropriate business terms, data classes and quality assessments so it can be discovered, governed and utilized effectively. Public API. See the official public APIs.

Metadata 56

Metadata Machine Learning Data Quality Statistics 56

AWS Big Data

JULY 18, 2024

The current method is largely manual, relying on emails and general communication, which not only increases overhead but also varies from one use case to another in terms of data governance. Amazon DataZone projects enable collaboration with teams through data assets and the ability to manage and monitor data assets across projects.

Data Lake 122

Data Lake Publishing Metadata Data-driven 122

erwin

JANUARY 23, 2025

IDC, BARC, and Gartner are just a few analyst firms producing annual or bi-annual market assessments for their research subscribers in software categories ranging from data intelligence platforms and data catalogs to data governance, data quality, metadata management and more.

Metadata 52

Metadata Data Quality Data Governance Software 52

AWS Big Data

JANUARY 18, 2024

AWS Lake Formation and the AWS Glue Data Catalog form an integral part of a data governance solution for data lakes built on Amazon Simple Storage Service (Amazon S3) with multiple AWS analytics services integrating with them. In 2022 , we talked about the enhancements we had done to these services. Crawlers, salut!

Data Lake 105

Data Lake Metadata Data Governance Statistics 105

Alation

NOVEMBER 22, 2022

What Is Data Governance In The Public Sector? Effective data governance for the public sector enables entities to ensure data quality, enhance security, protect privacy, and meet compliance requirements. With so much focus on compliance, democratizing data for self-service analytics can present a challenge.

Data Governance 52

Data Governance Metadata Data-driven Unstructured Data 52

AWS Big Data

MAY 7, 2024

Data governance is a key enabler for teams adopting a data-driven culture and operational model to drive innovation with data. Amazon DataZone allows you to simply and securely govern end-to-end data assets stored in your Amazon Redshift data warehouses or data lakes cataloged with the AWS Glue data catalog.

Metadata 87

Metadata Data Lake Data Processing Data-driven 87

Cloudera

OCTOBER 7, 2021

Application Logic: Application logic refers to the type of data processing, and can be anything from analytical or operational systems to data pipelines that ingest data inputs, apply transformations based on some business logic and produce data outputs. Key Design Principles of a Data Mesh.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

IBM Big Data Hub

MARCH 25, 2022

There are many different approaches, but you’ll want an architecture that can be used regardless of your data estate. In other words, the obstacles of data access, data integration and data protection are minimized, rendering maximum flexibility to the end users. Protection is applied on each data pipeline.

Metadata 104

Metadata Data Quality Data Governance Data Integration 104

AWS Big Data

MAY 1, 2024

Solution overview OneData defines three personas: Publisher – This role includes the organizational and management team of systems that serve as data sources. Responsibilities include: Load raw data from the data source system at the appropriate frequency. It is crucial in data governance and data management.

Dashboards 122

Dashboards Publishing Data-driven Cost-Benefit 122

Alation

APRIL 6, 2023

Data governance is traditionally applied to structured data assets that are most often found in databases and information systems. This blog focuses on governing spreadsheets that contain data, information, and metadata, and must themselves be governed. Simply put, metadata adds context.

Data Governance 52

Data Governance Metadata Cost-Benefit Structured Data 52

TDAN

OCTOBER 19, 2021

I last published my Data Governance Bill of “Rights” in a TDAN.com article circa 2017. I mentioned in the earlier piece that Data Governance is all about doing the “right” thing when it comes to managing your data. It’s all in the data. That seems like a long time ago.

Data Governance 52

Data Governance Publishing Management IT 52

AWS Big Data

FEBRUARY 26, 2025

S3 Tables integration with the AWS Glue Data Catalog is in preview, allowing you to stream, query, and visualize dataincluding Amazon S3 Metadata tablesusing AWS analytics services such as Amazon Data Firehose , Amazon Athena , Amazon Redshift, Amazon EMR, and Amazon QuickSight. With AWS Glue 5.0,

Analytics 59

Analytics Data Lake Metadata Data Warehouse 59

AWS Big Data

JULY 29, 2024

Amazon DataZone is a powerful data management service that empowers data engineers, data scientists, product managers, analysts, and business users to seamlessly catalog, discover, analyze, and govern data across organizational boundaries, AWS accounts, data lakes, and data warehouses.

Data Warehouse 91

Data Warehouse Sales Metadata Publishing 91

Alation

JANUARY 26, 2023

These data requirements could be satisfied with a strong data governance strategy. Governance can — and should — be the responsibility of every data user, though how that’s achieved will depend on the role within the organization. How can data engineers address these challenges directly?

Data Governance 52

Data Governance Strategy Data Quality Marketing 52

TDAN

APRIL 20, 2021

I published an article a few months back that was titled Where Does Data Governance Fit in a Data Strategy (and other important questions). In the article, I quickly outlined seven primary elements of a data strategy as an answer to one of the “other important questions.”

Data Strategy 98

Data Strategy Strategy Data Governance Publishing 98

AWS Big Data

JUNE 10, 2024

In this post, we delve into the key aspects of using Amazon EMR for modern data management, covering topics such as data governance, data mesh deployment, and streamlined data discovery. Organizations have multiple Hive data warehouses across EMR clusters, where the metadata gets generated.

Data Lake 96

Data Lake Metadata Data Warehouse Data Processing 96