erwin

FEBRUARY 6, 2020

As organizations deal with managing ever more data, the need to automate data management becomes clear. Last week erwin issued its 2020 State of Data Governance and Automation (DGA) Report. One piece of the research that stuck with me is that 70% of respondents spend 10 or more hours per week on data-related activities.

Management 144

Management Data Governance Cost-Benefit Metadata 144

Octopai

FEBRUARY 12, 2020

Like any good puzzle, metadata management comes with a lot of complex variables. That’s why you need to use data dictionary tools, which can help organize your metadata into an archive that can be navigated with ease and from which you can derive good information to power informed decision-making. Download Now.

Metadata 72

Metadata Enterprise Management Data Warehouse 72

Timo Elliott

JANUARY 6, 2022

If storage costs are escalating in a particular area, you may have found a good source of dark data. Analyze your metadata. If you’ve been properly managing your metadata as part of a broader data governance policy, you can use metadata management explorers to reveal silos of dark data in your landscape.

IT 135

IT Metadata Data-driven Data Governance 135

O'Reilly on Data

DECEMBER 19, 2018

For any codebase, it can tell you where the code came from (provenance), and all the changes that led from the original commit to the version you downloaded. If you suddenly see unexpected patterns in your social data, that may mean adversaries are attempting to poison your data sources.

Machine Learning 279

Machine Learning Software Metadata Testing 279

Octopai

SEPTEMBER 14, 2020

Modern data processing depends on metadata management to power enhanced business intelligence. Metadata is of course the information about the data, and the process of managing it is mysterious to those not trained in advanced BI. In this article, you will learn: What does metadata management do?

Metadata 52

Metadata Management Data Processing Machine Learning 52

Octopai

APRIL 19, 2021

Aptly named, metadata management is the process in which BI and Analytics teams manage metadata, which is the data that describes other data. In other words, data is the context and metadata is the content. Without metadata, BI teams are unable to understand the data’s full story.

Metadata 59

Metadata Management Business Intelligence Data Governance 59

AWS Big Data

JULY 8, 2024

Introduction to OpenLineage compatible data lineage The need to capture data lineage consistently across various analytical services and combine them into a unified object model is key in uncovering insights from the lineage artifact. Now let’s harvest the lineage metadata using CloudShell. Choose Run. We use version 1.9.1,

Visualization 119

Visualization Metadata Publishing Sales 119

erwin

JANUARY 23, 2025

IDC, BARC, and Gartner are just a few analyst firms producing annual or bi-annual market assessments for their research subscribers in software categories ranging from data intelligence platforms and data catalogs to data governance, data quality, metadata management and more.

Metadata 52

Metadata Data Quality Data Governance Software 52

Octopai

JANUARY 12, 2020

From Data Analysts, BI Architects, Developers, and Data Governance Leaders to IT and BI Managers. If you’re in the Business Intelligence world, there is a chance that you and I had a brief chat about metadata. Here’s what I found: Data always tells the truth, even when it lies. Inaccuracies in reports.”

Metadata 83

Metadata Management Data Governance Business Intelligence 83

Alation

MARCH 2, 2023

Analysis, however, requires enterprises to find and collect metadata. This data about data is valuable. It tells users which assets are most popular, how they’re used, properly governed, powerfully combined, and much more. What Is Active Metadata Management?

Metadata 52

Metadata Marketing IT Data Quality 52

erwin

OCTOBER 4, 2021

Metadata management is essential to becoming a data-driven organization and reaping the competitive advantage your organization’s data offers. Gartner refers to metadata as data that is used to enhance the usability, comprehension, utility or functionality of any other data point. How the data has changed.

Metadata 52

Metadata Software Management Visualization 52

AWS Big Data

JUNE 10, 2024

In this post, we delve into the key aspects of using Amazon EMR for modern data management, covering topics such as data governance, data mesh deployment, and streamlined data discovery. Organizations have multiple Hive data warehouses across EMR clusters, where the metadata gets generated.

Data Lake 116

Data Lake Metadata Data Warehouse Data Processing 116

AWS Big Data

DECEMBER 9, 2024

For our heater example, Icebergs change log view would allow us to effortlessly retrieve a timeline of all price changes, complete with timestamps and other relevant metadata, as shown in the following table. Select the following JAR files from the Iceberg releases page and download these JAR files on your local machine: 1.6.1

Snapshot 64

Snapshot Data Warehouse Data Lake Data Quality 64

Octopai

OCTOBER 28, 2020

As you may have experienced, one of the many impacts of the COVID-19 pandemic is that on-premises metadata management may no longer be a feasible way to connect BI teams. Further, although key stakeholders may be physically away from the workplace, they still need to access metadata information to produce reports.

Metadata 52

Metadata Management Cost-Benefit Business Intelligence 52

AWS Big Data

MAY 7, 2024

Data governance is a key enabler for teams adopting a data-driven culture and operational model to drive innovation with data. Amazon DataZone allows you to simply and securely govern end-to-end data assets stored in your Amazon Redshift data warehouses or data lakes cataloged with the AWS Glue data catalog.

Metadata 107

Metadata Data Lake Data Processing Data-driven 107

Alation

NOVEMBER 11, 2021

As IT leaders oversee migration, it’s critical they do not overlook data governance. Data governance is essential because it ensures people can access useful, high-quality data. Therefore, the question is not if a business should implement cloud data management and governance, but which framework is best for them.

Data Governance 52

Data Governance Data Quality Metrics Metadata 52

erwin

MARCH 18, 2021

The e-guide takes a deep dive into the evolving role of CDOs at financial organizations, tapping into the minds of 100+ financial global financial leaders and C-suite executives to look at the latest trends and provide a roadmap for developing an offensive data management strategy. Data Governance Is Not Just About Compliance.

Management 82

Management Data Governance Metadata Risk 82

AWS Big Data

JUNE 26, 2024

Administrators can customize Amazon DataZone to use existing AWS resources, enabling Amazon DataZone portal users to have federated access to those AWS services to catalog, share, and subscribe to data, thereby establishing data governance across the platform.

Data Lake 119

Data Lake Data Warehouse Unstructured Data Data Governance 119

Octopai

APRIL 11, 2021

The DCIA is based on the Consumer Privacy Protection Act (CPPA), which sets out the regulations for consumer privacy, and the Personal Information and Data Tribunal Act, which sets out the framework for enforcing penalties for violations. The post The Digital Charter Implementation Act & Metadata Management appeared first on Octopai.

Metadata 52

Metadata Management Measurement IT 52

AWS Big Data

AUGUST 15, 2024

Amazon Redshift is a fast, scalable, and fully managed cloud data warehouse that allows you to process and run your complex SQL analytics workloads on structured and semi-structured data. Amazon DataZone natively supports data sharing for Amazon Redshift data assets. In the post_dq_results_to_datazone.py

Data Quality 111

Data Quality Visualization Metadata Key Performance Indicator 111

AWS Big Data

JULY 18, 2024

The current method is largely manual, relying on emails and general communication, which not only increases overhead but also varies from one use case to another in terms of data governance. Data domain producers publish data assets using datasource run to Amazon DataZone in the Central Governance account.

Data Lake 130

Data Lake Publishing Metadata Data-driven 130

AWS Big Data

MARCH 7, 2023

Analytics reference architecture for gaming organizations In this section, we discuss how gaming organizations can use a data hub architecture to address the analytical needs of an enterprise, which requires the same data at multiple levels of granularity and different formats, and is standardized for faster consumption.

Analytics 97

Analytics Data Warehouse Data Lake Metadata 97

IBM Big Data Hub

FEBRUARY 26, 2024

The financial services industry has been in the process of modernizing its data governance for more than a decade. But as we inch closer to global economic downturn, the need for top-notch governance has become increasingly urgent. The answer is data lineage. The Basel Committee released BCBS 239 as far back as 2013.

Cost-Benefit 90

Cost-Benefit Metadata Data Governance Reporting 90

erwin

MARCH 18, 2021

The e-guide takes a deep dive into the evolving role of CDOs at financial organizations, tapping into the minds of 100+ financial global financial leaders and C-suite executives to look at the latest trends and provide a roadmap for developing an offensive data management strategy. Data Governance Is Not Just About Compliance.

Management 64

Management Data Governance Metadata Risk 64

Octopai

JANUARY 31, 2021

The particular episode we recommend looks at how WeWork struggled with understanding their data lineage so they created a metadata repository to increase visibility. Agile Data. Another podcast we think is worth a listen is Agile Data. In terms of data lineage, they offer an excellent definition here. .

Data Governance 83

Data Governance Data Processing Data Quality Metadata 83

Alation

FEBRUARY 13, 2020

Why do we need a data catalog? What does a data catalog do? These are all good questions and a logical place to start your data cataloging journey. Data catalogs have become the standard for metadata management in the age of big data and self-service analytics. Figure 1 – Data Catalog Metadata Subjects.

Metadata 64

Metadata Data Lake Recreation/Entertainment Big Data 64

Octopai

MAY 5, 2020

They collected insights from 293 business and IT professionals across a wide range of industries, shining light on why automation metadata operations and the business glossary are priorities for organizations today. Use Case #2: Daily Data Operations. Another BI challenge comes from everyday metadata management operations.

Metadata 52

Metadata Reporting Data Warehouse Management 52

Octopai

OCTOBER 8, 2020

Check out our white paper "Do I Need a Data Dictionary, Business Glossary or Catalog?" Download the White Paper. A Data Catalog (BI Catalog) Manages All of Your Reports. An enterprise data catalog is a tool that provides an inventory of all reports an organization has, the content of these reports, and how its data is stored.

Metadata 98

Metadata Reporting Business Intelligence Management 98

Alation

FEBRUARY 13, 2020

More recently we’ve started to use the term to describe managed collections of many kinds such as curated content at a website, curated music and videos available through streaming services, and curated apps through download services. Data curation is a metadata management activity and data catalogs are essential data curation technology.

Metadata 52

Metadata Data Warehouse Data Lake Data Governance 52

Alation

MARCH 11, 2022

This white paper makes this information actionable with a methodology, so you can learn how to implement a meshy fabric with your data catalog. For the full story, download the white paper here ! It will offload pressure from IT , improve your data supply chain, and lead to smarter decision making. Download it today.

Metadata 52

Metadata Data Governance ROI Data Quality 52

Alation

SEPTEMBER 1, 2022

Download the full report. The firm focuses on business software that supports business intelligence, analytics, data management, and other key data areas. Activating metadata within a data intelligence platform surfaces key insights and unburden experts from tedious manual work. The solution? Is this asset popular?

Reporting 52

Reporting Metadata Data Governance Interactive 52

erwin

FEBRUARY 28, 2024

One of the key ingredients to ensure data is really embedded in an organization, and one of the key enablers to increase the strategic impact of data, is the setup of a successful data governance program. Technology is an enabler, and for data governance this is essentially having an excellent metadata management tool.

Metadata 52

Metadata Data Governance Data Quality Technology 52

Alation

DECEMBER 13, 2022

Today a modern catalog hosts a wide range of users (like business leaders, data scientists and engineers) and supports an even wider set of use cases (like data governance , self-service , and cloud migration ). So feckless buyers may resort to buying separate data catalogs for use cases like…. Data governance.

Metadata 52

Metadata Data Governance Enterprise Data-driven 52

AWS Big Data

MARCH 10, 2023

Apache Iceberg overview Iceberg is an open-source table format that brings the power of SQL tables to big data files. It enables ACID transactions on tables, allowing for concurrent data ingestion, updates, and queries, all while using familiar SQL. The Iceberg table is synced with the AWS Glue Data Catalog. and update-item.py.

Data Lake 122

Data Lake Sales Data Warehouse Snapshot 122

erwin

FEBRUARY 12, 2021

For example, if tomorrow you decide you want ISO (International Organization for Standardization) 27001 compliance for your information security management system, you can go to the appropriate ISO site, download the entire standard with all the assessments with all the descriptions, mandates, questions and documents that you will need to provide.

Risk 98

Risk Slice and Dice Management Enterprise 98

Alation

SEPTEMBER 22, 2022

Overseeing data quality and ensuring proper usage represent two core reasons. Data pipelines contain valuable information that can be used to improve data quality and ensure data is used properly. Data Pipeline Strategy. With Fivetran’s Metadata API, a valuable set of data is now part of the equation.

Metadata 52

Metadata Data-driven Data Quality Visualization 52

Alation

AUGUST 16, 2022

This report underscores the growing need at enterprises for a catalog to drive key use cases, including self-service BI , data governance , and cloud data migration. You can download a copy of the report here. They rave about how they’ve achieved greater success with data governance using Alation.

Data Quality 52

Data Quality Data Governance Metadata Reporting 52