TDAN

DECEMBER 3, 2019

Recently, I was giving a presentation and someone asked me which segment of “the DAMA wheel” did I think semantics most affected. I said I thought it affected all of them pretty profoundly, but perhaps the Metadata wedge the most. I thought I’d spend a bit of time to reflect on the question and answer […].

Metadata 88

Metadata Data Architecture IT Data Governance 88

Jen Stirrup

MARCH 29, 2023

SAP helps to solve this search problem by offering ways to simplify business data with a solid data foundation that powers SAP Datasphere. It fits neatly with the renewed interest in data architecture, particularly data fabric architecture. They fail to get a grip on their data.

Data Warehouse 121

Data Warehouse Metadata Data Integration Business Intelligence 121

AWS Big Data

JUNE 10, 2024

Organizations have multiple Hive data warehouses across EMR clusters, where the metadata gets generated. To address this challenge, organizations can deploy a data mesh using AWS Lake Formation that connects the multiple EMR clusters. An entity can act both as a producer of data assets and as a consumer of data assets.

Data Lake 116

Data Lake Metadata Data Warehouse Data Processing 116

CIO Business Intelligence

JUNE 23, 2022

The cause is hybrid data – the massive amounts of data created everywhere businesses operate – in clouds, on-prem, and at the edge. Only a fraction of data created is actually stored and managed, with analysts estimating it to be between 4 – 6 ZB in 2020. Clearly, hybrid data presents a massive opportunity and a tough challenge.

IT 97

IT Data Architecture Unstructured Data Big Data 97

AWS Big Data

SEPTEMBER 7, 2023

With exponential growth in data volume, centralized monitoring becomes challenging. It is also crucial to audit granular data access for security and compliance needs. This blog post presents an architecture solution that allows customers to extract key insights from Amazon S3 access logs at scale.

Metadata 116

Metadata Dashboards Metrics Visualization 116

Ontotext

DECEMBER 1, 2023

The event held the space for presentations, discussions, and one-on-one meetings, where more than 20 partners, 1064 Registrants from 41 countries, spanning across 25 industries came together. It was presented by Summit Pal, Strategic Technology Director at Ontotext and former Gartner VP Analyst.

Metadata 78

Metadata Sales Machine Learning Consulting 78

AWS Big Data

MARCH 7, 2024

At the same time, they need to optimize operational costs to unlock the value of this data for timely insights and do so with a consistent performance. With this massive data growth, data proliferation across your data stores, data warehouse, and data lakes can become equally challenging.

Data Lake 131

Data Lake Analytics Dashboards Metrics 131

AWS Big Data

NOVEMBER 16, 2023

Data Vault 2.0 allows for the following: Agile data warehouse development Parallel data ingestion A scalable approach to handle multiple data sources even on the same entity A high level of automation Historization Full lineage support However, Data Vault 2.0

Enterprise 111

Enterprise Data Warehouse Data Lake Optimization 111

AWS Big Data

DECEMBER 19, 2024

From establishing an enterprise-wide data inventory and improving data discoverability, to enabling decentralized data sharing and governance, Amazon DataZone has been a game changer for HEMA. HEMA has a bespoke enterprise architecture, built around the concept of services.

Data Governance 52

Data Governance Publishing Data-driven Metadata 52

Ontotext

JANUARY 10, 2024

Limiting growth by (data integration) complexity Most operational IT systems in an enterprise have been developed to serve a single business function and they use the simplest possible model for this. In both cases, semantic metadata is the glue that turns knowledge graphs into hubs of data, metadata, and content.

Metadata 72

Metadata Slice and Dice Data Integration Enterprise 72

AWS Big Data

OCTOBER 1, 2024

The following graph illustrates these runtime improvements for the full benchmark (all TPC-DS queries) over the past year, including the additional boost from using AWS Glue Data Catalog column statistics. This can have a significant impact on overall query performance.

Data Lake 115

Data Lake Statistics Broadcasting Optimization 115

AWS Big Data

JANUARY 22, 2024

Cost and resource efficiency – This is an area where Acast observed a reduction in data duplication, and therefore cost reduction (in some accounts, removing the copy of data 100%), by reading data across accounts while enabling scaling. Some examples of Acast’s domains are presented in the following figure.

Data-driven 118

Data-driven Advertising Metadata Data Architecture 118

AWS Big Data

NOVEMBER 15, 2023

It seamlessly consolidates data from various data sources within AWS, including AWS Cost Explorer (and forecasting with Cost Explorer ), AWS Trusted Advisor , and AWS Compute Optimizer. Data providers and consumers are the two fundamental users of a CDH dataset. These ingested datasets are used as a source in CLEA dashboards.

Analytics 111

Analytics Dashboards Metadata Data Warehouse 111

AWS Big Data

APRIL 19, 2023

Independent data products often only have value if you can connect them, join them, and correlate them to create a higher order data product that creates additional insights. A modern data architecture is critical in order to become a data-driven organization. We focus on the former.

Technology 105

Technology Data-driven Machine Learning Sales 105

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. The File Manager Lambda function consumes those messages, parses the metadata, and inserts the metadata to the DynamoDB table odpf_file_tracker.

Data Lake 111

Data Lake Data Processing Metadata Snapshot 111

AWS Big Data

JANUARY 31, 2023

In this post, we are excited to summarize the features that the AWS Glue Data Catalog, AWS Glue crawler, and Lake Formation teams delivered in 2022. Whether you are a data platform builder, data engineer, data scientist, or any technology leader interested in data lake solutions, this post is for you.

Data Lake 98

Data Lake Data Governance Data Architecture Machine Learning 98

AWS Big Data

APRIL 3, 2024

Today, we are pleased to announce that Amazon DataZone is now able to present data quality information for data assets. If the asset has AWS Glue Data Quality enabled, you can now quickly visualize the data quality score directly in the catalog search pane.

Data Quality 119

Data Quality Visualization Metadata Metrics 119

AWS Big Data

SEPTEMBER 6, 2023

In this post, we aim to address this issue and present how you can use Amazon API Gateway and AWS Lambda to navigate around this obstacle. He works with enterprise FSI customers and is primarily specialized in machine learning and data architectures. Daniel Wessendorf is a Global Solutions Architect at AWS based in Munich.

Testing 116

Testing Metadata Cost-Benefit Internet of Things 116

Cloudera

SEPTEMBER 15, 2022

The consumption of the data should be supported through an elastic delivery layer that aligns with demand, but also provides the flexibility to present the data in a physical format that aligns with the analytic application, ranging from the more traditional data warehouse view to a graph view in support of relationship analysis.

Data Lake 83

Data Lake Data Architecture Data-driven Data Warehouse 83

AWS Big Data

JULY 20, 2023

With data becoming the driving force behind many industries today, having a modern data architecture is pivotal for organizations to be successful. We explore why Orca chose to build a transactional data lake and examine the key considerations that guided the selection of Apache Iceberg as the preferred table format.

Data Lake 98

Data Lake Analytics Snapshot Data Quality 98

Alation

MARCH 31, 2023

Ehtisham Zaidi, Gartner’s VP of data management, and Robert Thanaraj, Gartner’s director of data management, gave an update on the fabric versus mesh debate in light of what they call the “active metadata era” we’re currently in. The foundations of successful data governance The state of data governance was also top of mind.

Data Analytics 52

Data Analytics Analytics Metadata Data Governance 52

Alation

JULY 7, 2022

In 2013 I joined American Family Insurance as a metadata analyst. I had always been fascinated by how people find, organize, and access information, so a metadata management role after school was a natural choice. The use cases for metadata are boundless, offering opportunities for innovation in every sector.

Metadata 52

Metadata Data-driven Insurance Statistics 52

Alation

FEBRUARY 20, 2020

In fact, Wells has identified four characteristics of digital marketplaces that should be present in any EDW. Categorization organizes the marketplace to simplify browsing (either by data asset type or topic). Through this ongoing feedback loop, the quality of the data in the marketplace undergoes continuous improvement.

Data Warehouse 52

Data Warehouse Metadata Data Lake Data Architecture 52

Ontotext

NOVEMBER 16, 2023

Figure 1 Shows the overall idea of a data mesh with the major components: What Is a Data Mesh and How Does It Work? Think of data mesh as an operational mode for organizations with a domain-driven, decentralized data architecture. What Is a Data Product and Who Owns Them?

Metadata 52

Metadata Data-driven Data Quality Data Architecture 52

IBM Big Data Hub

JUNE 30, 2022

The diversity of data types, data processing, integration and consumption patterns used by organizations has grown exponentially. Extend data governance to foster trust in your data by creating transparency, eliminating bias and ensuring explainability for data and insights fueled by machine learning and AI.

Data Strategy 56

Data Strategy Strategy Business Objectives Data-driven 56

erwin

FEBRUARY 28, 2024

While the essence of success in data governance is people and not technology, having the right tools at your fingertips is crucial. Technology is an enabler, and for data governance this is essentially having an excellent metadata management tool. Next to data governance, data architecture is really embedded in our DNA.

Metadata 52

Metadata Data Governance Data Quality Technology 52

TDAN

MAY 2, 2023

In today’s data-driven world, organizations are demanding and consuming vast amounts of data — data that needs to be easily accessed, analyzed, and presented in a way that enables quick action.

Data-driven 59

Data-driven IT Data Architecture Metadata 59

TDAN

JULY 5, 2022

Whichever metaphor you would like to use, what is certain is that no organization will survive the twenty-first century without optimizing the use of its data assets. Similarly, cybersecurity, privacy, and compliance risks increasingly present huge […].

Data-driven 52

Data-driven Optimization Risk Enterprise 52

AWS Big Data

MARCH 3, 2023

split(";") # Exit if length of table names and partition keys are different to ensure data is provided for all tables. if partition key is not present enter empty semicolon - T1_PK;;T3PK") sys.exit(0) i = 0 while i < len(tables): table = tables[i] partition_key = partition_keys[i].split(",") if len(tables)!=len(partition_keys):

Data Lake 98

Data Lake Dashboards Metrics Metadata 98

Andrew White

JANUARY 9, 2022

Check this out: The Foundation of an Effective Data and Analytics Operating Model — Presentation Materials. Most of D&A concerns and activities are done within EA in the Info/Data architecture domain/phases. – Data (and analytics) governance remains a challenge. Great presentation, thank you.

Analytics 64

Analytics Measurement Data-driven Modeling 64

Ontotext

APRIL 15, 2020

An example of the sort of linked data reasoning that can be employed here is that if quarantine and social distancing measures are in place for a region, then a community that’s part of this region will be subject to those same restrictions, so you don’t need to materialize everything in the graph. To Sum It Up.

Publishing 105

Publishing Metadata Data mining Data Architecture 105

Jet Global

MAY 1, 2023

Discuss, don’t present. Present your business case. To support your case, present findings from the State of Embedded Analytics study. Information Delivery The main reason software providers take on an embedded analytics project is to improve how data is presented. It is now most definitely a need-to-have.

Analytics 52

Analytics Cost-Benefit Visualization Dashboards 52

AWS Big Data

DECEMBER 20, 2024

Jumia is a technology company born in 2012, present in 14 African countries, with its main headquarters in Lagos, Nigeria. Jumia is present in NYSE and has a market cap of $554 million. These phases are: data orchestration, data migration, data ingestion, data processing, and data maintenance.

Metadata 119

Metadata Data-driven Snapshot Data Lake 119

AWS Big Data

JANUARY 23, 2025

Amazon SageMaker Lakehouse enables a unified, open, and secure lakehouse platform on your existing data lakes and warehouses. Its unified data architecture supports data analysis, business intelligence, machine learning, and generative AI applications, which can now take advantage of a single authoritative copy of data.

Data Lake 122

Data Lake Data Warehouse Metadata Machine Learning 122

AWS Big Data

APRIL 28, 2025

The data is stored in Apache Parquet format with AWS Glue Catalog providing metadata management. While this architecture supported NI analytical needs, it lacked the flexibility required for a truly open and adaptable data platform. This meant NI couldnt rely on Glue Catalog events to detect partition changes.

Data Lake 69

Data Lake Metadata Cost-Benefit Snapshot 69

AWS Big Data

DECEMBER 2, 2024

The rapid adoption has enabled them to quickly streamline operations, enhance collaboration, and gain more accessible, scalable solutions for managing their critical data and workflows. AWS Glue also supports the ability to apply complex data transformations, enabling efficient data integration and preparation to meet your needs.

Data Lake 105

Data Lake Testing Data Integration Metadata 105

AWS Big Data

MAY 12, 2025

Custom images presented the following challenges: Custom images are supported as of version 6.9.0, Integrating lineage into EMR Serverless AppsFlyer developed a robust solution for column-level lineage collection to provide comprehensive visibility into data transformations across pipelines.

Metrics 81

Metrics Cost-Benefit Metadata Data Lake 81