O'Reilly on Data

MAY 21, 2019

Use ML to unlock new data types—e.g., Consider deep learning, a specific form of machine learning that resurfaced in 2011/2012 due to record-setting models in speech and computer vision. Not surprisingly, data integration and ETL were among the top responses, with 60% currently building or evaluating solutions in this area.

Machine Learning 279

Machine Learning Technology Deep Learning Data Science 279

AWS Big Data

JUNE 10, 2024

In this post, we delve into the key aspects of using Amazon EMR for modern data management, covering topics such as data governance, data mesh deployment, and streamlined data discovery. Organizations have multiple Hive data warehouses across EMR clusters, where the metadata gets generated.

Data Lake 116

Data Lake Metadata Data Warehouse Data Processing 116

AWS Big Data

AUGUST 15, 2024

With Amazon DataZone, individual business units can discover and directly consume these new data assets, gaining insights to a holistic view of the data (360-degree insights) across the organization. The Central IT team manages a unified Redshift data warehouse, handling all data integration, processing, and maintenance.

Data Lake 121

Data Lake Data Warehouse Data Governance Publishing 121

Domino Data Lab

APRIL 3, 2019

Paco Nathan ‘s latest column dives into data governance. This month’s article features updates from one of the early data conferences of the year, Strata Data Conference – which was held just last week in San Francisco. In particular, here’s my Strata SF talk “Overview of Data Governance” presented in article form.

Machine Learning 84

Machine Learning Data Governance Metadata Data Science 84

Alation

DECEMBER 15, 2022

December 2012: Alation forms and goes to work creating the first enterprise data catalog. Later, in its inaugural report on data catalogs, Forrester Research recognizes that “Alation started the MLDC trend.”. May 2016: Alation named a Gartner Cool Vendor in their Data Integration and Data Quality, 2016 report.

Metadata 52

Metadata Data Governance Data Quality Marketing 52

AWS Big Data

JANUARY 30, 2023

On the AWS Glue console, under Data Integration and ETL in the navigation pane, choose Jobs. load("s3://"+ args['s3_bucket']+"/fullload/") sdf.printSchema() # Write data as DELTA TABLE sdf.write.format("delta").mode("overwrite").save("s3://"+ Vivek Singh is Senior Solutions Architect with the AWS Data Lab team.

Insurance 98

Insurance Data Lake Data-driven Analytics 98

AWS Big Data

NOVEMBER 15, 2024

By harnessing the capabilities of generative AI, you can automate the generation of comprehensive metadata descriptions for your data assets based on their documentation, enhancing discoverability, understanding, and the overall data governance within your AWS Cloud environment. The following is an example policy.

Metadata 118

Metadata Modeling Data-driven Machine Learning 118