erwin

OCTOBER 31, 2019

The Regulatory Rationale for Integrating Data Management & Data Governance. Now, as Cybersecurity Awareness Month comes to a close – and ghosts and goblins roam the streets – we thought it a good time to resurrect some guidance on how data governance can make data security less scary.

Data Governance 105

Data Governance Metadata Risk Data Lake 105

erwin

APRIL 2, 2021

How can companies protect their enterprise data assets, while also ensuring their availability to stewards and consumers while minimizing costs and meeting data privacy requirements? Data Security Starts with Data Governance. Lack of a solid data governance foundation increases the risk of data-security incidents.

Data Governance 106

Data Governance Cost-Benefit Metadata Risk 106

erwin

DECEMBER 13, 2018

This past year witnessed a data governance awakening – or as the Wall Street Journal called it, a “global data governance reckoning.” There was tremendous data drama and resulting trauma – from Facebook to Equifax and from Yahoo to Marriott. So what’s on the horizon for data governance in the year ahead?

Data Governance 91

Data Governance IoT Internet of Things Data-driven 91

AWS Big Data

DECEMBER 12, 2024

The combination of these three services provides a powerful, comprehensive solution for end-to-end data lineage analysis. In this post, we use dbt for data modeling on both Amazon Athena and Amazon Redshift. For detailed steps on creating and configuring Lambda Layers , see the AWS Lambda Layers documentation.

Snapshot 98

Snapshot Recreation/Entertainment Experimentation Data Lake 98

Cloudera

AUGUST 4, 2021

And if data security tops IT concerns, data governance should be their second priority. Not only is it critical to protect data, but data governance is also the foundation for data-driven businesses and maximizing value from data analytics. But it’s still not easy. But it’s still not easy.

Data Governance 86

Data Governance Metadata Data-driven Enterprise 86

AWS Big Data

JULY 29, 2024

In the era of big data, data lakes have emerged as a cornerstone for storing vast amounts of raw data in its native format. They support structured, semi-structured, and unstructured data, offering a flexible and scalable environment for data ingestion from multiple sources.

Metadata 126

Metadata Snapshot Data Lake Metrics 126

CIO Business Intelligence

OCTOBER 2, 2024

And the other is retrieval augmented generation (RAG) models, where pieces of data from a larger source are vectorized to allow users to “talk” to the data. For example, they can take a thousand-page document, have it ingested by the model, and then ask the model questions about it. For us, it’s all part of data governance.

Management 141

Management Data Governance Cost-Benefit Structured Data 141

AWS Big Data

DECEMBER 4, 2024

In today’s data-driven world , organizations are constantly seeking efficient ways to process and analyze vast amounts of information across data lakes and warehouses. This post will showcase how this data can also be queried by other data teams using Amazon Athena. Verify that you have Python version 3.7

Data Lake 116

Data Lake Metadata Insurance Data-driven 116

AWS Big Data

FEBRUARY 26, 2025

Analytics remained one of the key focus areas this year, with significant updates and innovations aimed at helping businesses harness their data more efficiently and accelerate insights. From enhancing data lakes to empowering AI-driven analytics, AWS unveiled new tools and services that are set to shape the future of data and analytics.

Analytics 62

Analytics Data Lake Metadata Data Warehouse 62

BI-Survey

AUGUST 21, 2020

Leaders rely less on data mart deployment than on lean, flexible architectures and usable data based on cloud services, a complementary data lake, data governance, data hubs and data catalogs. They are opting for cloud data services more frequently. Data must become a C-level priority.

Data Warehouse 98

Data Warehouse Data Lake Data Governance Data Architecture 98

AWS Big Data

OCTOBER 20, 2023

Today, we are pleased to announce new AWS Glue connectors for Azure Blob Storage and Azure Data Lake Storage that allow you to move data bi-directionally between Azure Blob Storage, Azure Data Lake Storage, and Amazon Simple Storage Service (Amazon S3). option("header","true").load("wasbs://yourblob@youraccountname.blob.core.windows.net/loadingtest-input/100mb")

Data Lake 111

Data Lake Big Data Data Warehouse Consulting 111

Alation

FEBRUARY 17, 2023

This past week, I had the pleasure of hosting Data Governance for Dummies author Jonathan Reichental for a fireside chat , along with Denise Swanson , Data Governance lead at Alation. Can you have proper data management without establishing a formal data governance program?

Data Governance 52

Data Governance Data Quality Metadata Cost-Benefit 52

erwin

JULY 17, 2019

The solution is data intelligence. It improves IT and business data literacy and knowledge, supporting enterprise data governance and business enablement. Organizations need a real-time, accurate picture of the metadata landscape to: Discover data – Identify and interrogate metadata from various data management silos.

Digital Transformation 98

Digital Transformation Strategy Metadata Data-driven 98

CIO Business Intelligence

OCTOBER 13, 2023

Solutions data architect: These individuals design and implement data solutions for specific business needs, including data warehouses, data marts, and data lakes. Application data architect: The application data architect designs and implements data models for specific software applications.

Data Architecture 132

Data Architecture Data Warehouse Statistics Visualization 132

AWS Big Data

MARCH 7, 2023

A data hub is a center of data exchange that constitutes a hub of data repositories and is supported by data engineering, data governance, security, and monitoring services. A data hub contains data at multiple levels of granularity and is often not integrated.

Analytics 97

Analytics Data Warehouse Data Lake Metadata 97

Alation

APRIL 6, 2023

Data governance is traditionally applied to structured data assets that are most often found in databases and information systems. Spreadsheets are not going away any time soon, so it makes sense to incorporate them into the data landscape. There are others that consider spreadsheets to be trouble.

Data Governance 52

Data Governance Metadata Cost-Benefit Structured Data 52

CIO Business Intelligence

AUGUST 14, 2024

Organizations with experience building enterprise data lakes connecting to many different data sources have AI advantages. They’re more able to connect to a diverse set of data sources, build more complex context around queries going through the model, and do retrieval augmented generation,” he adds.

Enterprise 136

Enterprise Cost-Benefit Experimentation Modeling 136

Alation

FEBRUARY 1, 2022

A new research report by Ventana Research, Embracing Modern Data Governance , shows that modern data governance programs can drive a significantly higher ROI in a much shorter time span. Historically, data governance has been a manual and restrictive process, making it almost impossible for these programs to succeed.

Data Lake 52

Data Lake Data Governance ROI Cost-Benefit 52

Cloudera

NOVEMBER 18, 2021

By adopting a custom developed application based on the Cloudera ecosystem, Carrefour has combined the legacy systems into one platform which provides access to customer data in a single data lake. In doing so, Bank of the West has modernized and centralized its Big Data platform in just one year.

Data Lake 77

Data Lake Cost-Benefit Digital Transformation Risk 77

erwin

JANUARY 11, 2019

Organizations have spent a lot of time and money trying to harmonize data across diverse platforms , including cleansing, uploading metadata, converting code, defining business glossaries, tracking data transformations and so on. But the attempts to standardize data across the entire enterprise haven’t produced the desired results.

Data Governance 40

Data Governance Risk Metadata Management 40

CIO Business Intelligence

SEPTEMBER 21, 2022

However, as data enablement platform, LiveRamp, has noted, CIOs are well across these requirements, and are now increasingly in a position where they can start to focus on enablement for people like the CMO. Successfully capitalising on the data opportunity requires a whole-of-business approach. Gaining Executive Buy-In.

Data Lake 105

Data Lake Risk Marketing Data Warehouse 105

AWS Big Data

AUGUST 13, 2024

With AWS Glue, you can discover and connect to hundreds of diverse data sources and manage your data in a centralized data catalog. It enables you to visually create, run, and monitor extract, transform, and load (ETL) pipelines to load data into your data lakes.

Data Processing 115

Data Processing Visualization Data Lake Data Processing 115

AWS Big Data

AUGUST 19, 2024

Refer to Enabling AWS PrivateLink in the Snowflake documentation to verify the steps, required access level, and service level to set the configurations. In summary, the native Snowflake connector and private connectivity model outlined here provide a performant, secure way to include Snowflake data in AWS big data workflows.

Analytics 119

Analytics Data-driven Data Integration Data Lake 119

IBM Big Data Hub

OCTOBER 12, 2023

Accounting for the complexities of the AI lifecycle Unfortunately, typical data storage and data governance tools fall short in the AI arena when it comes to helping an organization perform the tasks that underline efficient and responsible AI lifecycle management. And that makes sense.

Cost-Benefit 76

Cost-Benefit Metadata Data Governance Optimization 76

Ontotext

FEBRUARY 12, 2024

Determine ownership by making sure all teams involved in the data mesh own the quality of their domain data, ensure service-level agreements are met, and share that data with data contracts. Domain teams should continually monitor for data errors with data validation checks and incorporate data lineage to track usage.

Data Quality 69

Data Quality Data-driven Data Lake Data Governance 69

Cloudera

OCTOBER 7, 2021

Those decentralization efforts appeared under different monikers through time, e.g., data marts versus data warehousing implementations (a popular architectural debate in the era of structured data) then enterprise-wide data lakes versus smaller, typically BU-Specific, “data ponds”.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

Alation

FEBRUARY 13, 2020

Figure 1 illustrates the typical metadata subjects contained in a data catalog. Figure 1 – Data Catalog Metadata Subjects. Datasets are the files and tables that data workers need to find and access. They may reside in a data lake, warehouse, master data repository, or any other shared data resource.

Metadata 64

Metadata Data Lake Recreation/Entertainment Big Data 64

Domino Data Lab

APRIL 3, 2019

Paco Nathan ‘s latest column dives into data governance. This month’s article features updates from one of the early data conferences of the year, Strata Data Conference – which was held just last week in San Francisco. In particular, here’s my Strata SF talk “Overview of Data Governance” presented in article form.

Machine Learning 84

Machine Learning Data Governance Metadata Data Science 84

Alation

APRIL 14, 2022

Semantics, context, and how data is tracked and used mean even more as you stretch to reach post-migration goals. This is why, when data moves, it’s imperative for organizations to prioritize data discovery. Data discovery is also critical for data governance , which, when ineffective, can actually hinder organizational growth.

Metadata 59

Metadata Data Governance Data-driven Modeling 59

Sanjeev Mohan

JANUARY 15, 2018

You see, I write research documents, help my clients through inquiries and present at various events. However, each document that I have written is really a love letter to that topic. Somehow the data deluge barely leaves enough oxygen for a social media dopamine fix! We are done proving the value of the data lakes.

Data Lake 45

Data Lake IoT Big Data Interactive 45

Alation

JUNE 9, 2021

In this four-part blog series on data culture, we’re exploring what a data culture is and the benefits of building one, and then drilling down to explore each of the three pillars of data culture – data search & discovery, data literacy, and data governance – in more depth.

Data Governance 59

Data Governance Data-driven Machine Learning Data Lake 59

AWS Big Data

JULY 19, 2023

This connector provides access to Google Cloud Storage, facilitating cloud ETL processes for operational reporting, backup and disaster recovery, data governance, and more. This connector enables your data to be portable across Google Cloud Storage and Amazon S3. We welcome any feedback or questions in the comments section.

Big Data 98

Big Data Software Consulting Unstructured Data 98

AWS Big Data

JANUARY 22, 2024

Discussions with users showed they were happier to have faster access to data in a simpler way, a more structured data organization, and a clear mapping of who the producer is. A lot of progress has been made to advance their data-driven culture (data literacy, data sharing, and collaboration across business units).

Data-driven 118

Data-driven Advertising Metadata Data Architecture 118

Alation

APRIL 18, 2023

Top use cases for data profiling Data governance Data governance describes how data should be gathered and used within an organization, impacting data quality, data security, data privacy , and compliance. Do you need to define a data quality rule and add that to the profile?

IT 52

IT Data Quality Metadata Data Governance 52

Alation

MARCH 1, 2022

Airline Reporting Corporation (ARC) sells data products to travel agencies and airlines. Lineage helps them identify the source of bad data to fix the problem fast. Manual lineage will give ARC a fuller picture of how data was created between AWS S3 data lake, Snowflake cloud data warehouse and Tableau (and how it can be fixed).

Data Warehouse 52

Data Warehouse Data Lake Risk Data-driven 52

AWS Big Data

JULY 28, 2023

Data management and governance Addressing the challenges mentioned requires a combination of technical, operational, and legal measures. Organizations need to develop robust data governance practices, establish clear procedures for handling deletion requests, and maintain ongoing compliance with GDPR regulations.

Snapshot 98

Snapshot Metadata Measurement Data Warehouse 98

AWS Big Data

JULY 29, 2024

Amazon DataZone is a powerful data management service that empowers data engineers, data scientists, product managers, analysts, and business users to seamlessly catalog, discover, analyze, and govern data across organizational boundaries, AWS accounts, data lakes, and data warehouses.

Data Warehouse 113

Data Warehouse Sales Metadata Publishing 113