AWS Big Data

AUGUST 13, 2024

Amazon DataZone has announced a set of new data governance capabilities—domain units and authorization policies—that enable you to create business unit-level or team-level organization and manage policies according to your business needs. Some examples of child domain units are campaigns and events.

Data Governance 109

Data Governance Metadata Enterprise Sales 109

O'Reilly on Data

DECEMBER 19, 2018

If you suddenly see unexpected patterns in your social data, that may mean adversaries are attempting to poison your data sources. Anomaly detection may have originated in finance, but it is becoming a part of every data scientist’s toolkit. Tim Kraska on “How machine learning will accelerate data management systems”.

Machine Learning 279

Machine Learning Software Metadata Testing 279

AWS Big Data

OCTOBER 21, 2024

Business analysts enhance the data with business metadata/glossaries and publish the same as data assets or data products. The data security officer sets permissions in Amazon DataZone to allow users to access the data portal. Amazon Athena is used to query, and explore the data.

Sales 104

Sales Data-driven Data Processing Key Performance Indicator 104

erwin

APRIL 29, 2020

An understanding of the data’s origins and history helps answer questions about the origin of data in a Key Performance Indicator (KPI) reports, including: How the report tables and columns are defined in the metadata? Who are the data owners? Data lineage offers proof that the data provided is reflected accurately.

Metadata 111

Metadata Key Performance Indicator Data Governance Data Quality 111

Octopai

APRIL 19, 2021

Aptly named, metadata management is the process in which BI and Analytics teams manage metadata, which is the data that describes other data. In other words, data is the context and metadata is the content. Without metadata, BI teams are unable to understand the data’s full story. Dataconomy.

Metadata 59

Metadata Management Business Intelligence Data Governance 59

erwin

SEPTEMBER 17, 2020

Replace manual and recurring tasks for fast, reliable data lineage and overall data governance. It’s paramount that organizations understand the benefits of automating end-to-end data lineage. The importance of end-to-end data lineage is widely understood and ignoring it is risky business.

Cost-Benefit 143

Cost-Benefit Data Governance Metadata Reporting 143

AWS Big Data

JULY 8, 2024

We are excited to announce the preview of API-driven, OpenLineage-compatible data lineage in Amazon DataZone to help you capture, store, and visualize lineage of data movement and transformations of data assets on Amazon DataZone. The lineage visualized includes activities inside the Amazon DataZone business data catalog.

Visualization 108

Visualization Metadata Publishing Sales 108

erwin

APRIL 11, 2019

Fortunately, whenever the time comes, the first point of call will always be data governance, so organizations can prepare. Effective compliance with new data protection regulations requires a robust understanding of the “what, where and who” in terms of data and the stakeholders with access to it (i.e., employees).

Insurance 100

Insurance Data-driven Data Governance Metadata 100

Alation

MAY 31, 2023

The Data Governance & Information Quality Conference (DGIQ) is happening soon — and we’ll be onsite in San Diego from June 5-9. If you’re not familiar with DGIQ, it’s the world’s most comprehensive event dedicated to, you guessed it, data governance and information quality. The best part?

Data Governance 52

Data Governance Insurance Metadata Data-driven 52

Cloudera

OCTOBER 7, 2021

Application Logic: Application logic refers to the type of data processing, and can be anything from analytical or operational systems to data pipelines that ingest data inputs, apply transformations based on some business logic and produce data outputs.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

AWS Big Data

FEBRUARY 26, 2025

This premier event showcased groundbreaking advancements, keynotes from AWS leadership, hands-on technical sessions, and exciting product launches. Analytics remained one of the key focus areas this year, with significant updates and innovations aimed at helping businesses harness their data more efficiently and accelerate insights.

Analytics 59

Analytics Data Lake Metadata Data Warehouse 59

Octopai

JANUARY 12, 2020

From Data Analysts, BI Architects, Developers, and Data Governance Leaders to IT and BI Managers. If you’re in the Business Intelligence world, there is a chance that you and I had a brief chat about metadata. Here’s what I found: Data always tells the truth, even when it lies.

Metadata 83

Metadata Management Data Governance Business Intelligence 83

Octopai

FEBRUARY 9, 2023

Trying to manage data governance without a comprehensive data lineage solution can leave you feeling like your data keeps running away. It’s not easy to keep up with data and metadata on the move. A comprehensive data lineage tool is the secret weapon of successful data governance managers and data stewards.

Data Governance 52

Data Governance Metadata Data Quality Management 52

erwin

DECEMBER 9, 2024

Data observability provides the ability to immediately recognize, and be alerted to, the emergence of hallucinations and accept or reject these changes iteratively, thereby training and validating the data. Maybe your AI model monitors sales data, and the data is spiking for one region of the country due to a world event.

Metadata 52

Metadata Data Quality Sales Modeling 52

AWS Big Data

MAY 7, 2024

Data governance is a key enabler for teams adopting a data-driven culture and operational model to drive innovation with data. Amazon DataZone allows you to simply and securely govern end-to-end data assets stored in your Amazon Redshift data warehouses or data lakes cataloged with the AWS Glue data catalog.

Metadata 86

Metadata Data Lake Data Processing Data-driven 86

Cloudera

APRIL 18, 2024

Here are a couple of the biggest takeaways we had from our time at the event. In those discussions, it was clear that everyone understood the need to treat data estates more cohesively as a whole—that means bringing more attention to security, data governance, and metadata management, the latter of which has become increasingly popular.

Analytics 110

Analytics Metadata Data Strategy Optimization 110

AWS Big Data

JULY 18, 2024

The current method is largely manual, relying on emails and general communication, which not only increases overhead but also varies from one use case to another in terms of data governance. Data domain producers publish data assets using datasource run to Amazon DataZone in the Central Governance account.

Data Lake 122

Data Lake Publishing Metadata Data-driven 122

AWS Big Data

OCTOBER 4, 2023

Collaboration – Analysts, data scientists, and data engineers often own different steps within the end-to-end analytics journey but do not have an simple way to collaborate on the same governed data, using the tools of their choice.

Metadata 113

Metadata Data Lake Publishing Data Governance 113

Cloudera

AUGUST 10, 2021

SDX enhancements for improved platform and data governance, including the following notable features: . Atlas / Kafka integration provides metadata collection for Kafa producers/consumers so that consumers can manage, govern, and monitor Kafka metadata and metadata lineage in the Atlas UI. x, and 6.3.x,

Metadata 97

Metadata Sales Statistics Management 97

IBM Big Data Hub

NOVEMBER 21, 2023

Apache Kafka is a well-known open-source event store and stream processing platform and has grown to become the de facto standard for data streaming. Apache Kafka transfers data without validating the information in the messages. Kafka does not examine the metadata of your messages. What’s next?

Data Quality 96

Data Quality Metadata Data-driven Optimization 96

Domino Data Lab

JULY 2, 2019

In other words, using metadata about data science work to generate code. In this case, code gets generated for data preparation, where so much of the “time and labor” in data science work is concentrated. Less data gets decompressed, deserialized, loaded into memory, run through the processing, etc.

Metadata 105

Metadata Data Science Machine Learning Data-driven 105

AWS Big Data

MARCH 7, 2023

It covers how to use a conceptual, logical architecture for some of the most popular gaming industry use cases like event analysis, in-game purchase recommendations, measuring player satisfaction, telemetry data analysis, and more. A data hub contains data at multiple levels of granularity and is often not integrated.

Analytics 78

Analytics Data Warehouse Data Lake Metadata 78

AWS Big Data

APRIL 28, 2023

The first post of this series describes the overall architecture and how Novo Nordisk built a decentralized data mesh architecture, including Amazon Athena as the data query engine. The third post will show how end-users can consume data from their tool of choice, without compromising data governance.

Data Governance 72

Data Governance Management Data-driven Analytics 72

datapine

SEPTEMBER 29, 2022

This person (or group of individuals) ensures that the theory behind data quality is communicated to the development team. 2 – Data profiling. Data profiling is an essential process in the DQM lifecycle. Data monitoring and visualization: To be able to assess the quality of the data it is necessary to monitor it closely.

Data Quality 241

Data Quality Metrics Data-driven Management 241

Alation

JUNE 14, 2021

And third is what factors CIOs and CISOs should consider when evaluating a catalog – especially one used for data governance. The Role of the CISO in Data Governance and Security. They want CISOs putting in place the data governance needed to actively protect data. So CISOs must protect data.

Data Governance 97

Data Governance Recreation/Entertainment Data Lake Metadata 97

Cloudera

AUGUST 3, 2021

Advanced analytics and enterprise data are empowering several overarching initiatives in supply chain risk reduction – improved visibility and transparency into all aspects of the supply chain balanced with data governance and security. . Improve Visibility within Supply Chains.

Analytics 111

Analytics Digital Transformation Forecasting Risk 111

TDAN

MARCH 5, 2019

Getting business and leadership support for data governance programs – and building a data culture on that buy-in – remains a significant challenge in many organizations. The results of the new survey were presented at a Collibra event […].

Data Governance 40

Data Governance Metadata Data Strategy Strategy 40

AWS Big Data

JANUARY 24, 2023

AWS Lake Formation helps with enterprise data governance and is important for a data mesh architecture. It works with the AWS Glue Data Catalog to enforce data access and governance. This solution only replicates metadata in the Data Catalog, not the actual underlying data.

Data Architecture 72

Data Architecture Metadata Data Lake Snapshot 72

Alation

MAY 19, 2022

Alation attended last week’s Gartner Data and Analytics Summit in London from May 9 – 11, 2022. Coming off the heels of Data Innovation Summit in Stockholm, it’s clear that in-person events are back with a vengeance, and we’re thrilled about it. Establish what data you have. Leverage small data.

Metadata 52

Metadata Data Analytics Analytics Data Governance 52

AWS Big Data

SEPTEMBER 11, 2024

Source systems Aruba’s source repository includes data from three different operating regions in AMER, EMEA, and APJ, along with one worldwide (WW) data pipeline from varied sources like SAP S/4 HANA, Salesforce, Enterprise Data Warehouse (EDW), Enterprise Analytics Platform (EAP) SharePoint, and more.

Data Architecture 96

Data Architecture Optimization Data Warehouse Metadata 96

Alation

JUNE 1, 2023

So it’s fitting that Snowflake Summit , the premier event for data cloud strategy, will occur at Caesars Forum in Las Vegas on June 26–29 (togas not required). As a 2-time Snowflake Data Governance Partner of the Year , Alation knows how important this event is to the Snowflake community.

Metadata 52

Metadata Data Governance Data-driven Recreation/Entertainment 52

Andrew White

DECEMBER 9, 2019

The outline of the call went as follows: I was taking to a central state agency who was organizing a data governance initiative (in their words) across three other state agencies. All four agencies had reported an independent but identical experience with data governance in the past. Events or transactions.

Analytics 66

Analytics Data Lake Data Governance Data Warehouse 66

Alation

AUGUST 26, 2021

What Is Data Intelligence? Data intelligence is a system to deliver trustworthy, reliable data. It includes intelligence about data, or metadata. IDC coined the term, stating, “data intelligence helps organizations answer six fundamental questions about data.” Why keep data at all?

Metadata 64

Metadata Data Governance Dashboards Software 64

AWS Big Data

JUNE 9, 2023

Data in customers’ data lakes is used to fulfil a multitude of use cases, from real-time fraud detection for financial services companies, inventory and real-time marketing campaigns for retailers, or flight and hotel room availability for the hospitality industry. The following table summarizes the features.

Data Lake 125

Data Lake Metadata Statistics Optimization 125

AWS Big Data

JANUARY 31, 2023

Data governance is the collection of policies, processes, and systems that organizations use to ensure the quality and appropriate handling of their data throughout its lifecycle for the purpose of generating business value.

Data Lake 89

Data Lake Data Governance Data Architecture Machine Learning 89

AWS Big Data

JULY 24, 2024

Event tables: Fact or log-like tables contain events or actions your customers take. They often provide additional information to augment the data in event tables. For example, an entity table could be a product table that contains product metadata and joins to a transaction event table on a product_id.

Data Warehouse 110

Data Warehouse Cost-Benefit Marketing Testing 110

Octopai

JANUARY 31, 2021

The particular episode we recommend looks at how WeWork struggled with understanding their data lineage so they created a metadata repository to increase visibility. Agile Data. Another podcast we think is worth a listen is Agile Data. In terms of data lineage, they offer an excellent definition here. .

Data Governance 83

Data Governance Data Processing Data Quality Metadata 83