DataKitchen

DECEMBER 7, 2021

Back by popular demand, we’ve updated our data nerd Gift Giving Guide to cap off 2021. We’ve kept some classics and added some new titles that are sure to put a smile on your data nerd’s face. Fail Fast, Learn Faster: Lessons in Data-Driven Leadership in an Age of Disruption, Big Data, and AI, by Randy Bean.

Data-driven 246

Data-driven Data Governance Big Data Data Science 246

AWS Big Data

OCTOBER 29, 2024

To achieve this, they aimed to break down data silos and centralize data from various business units and countries into the BMW Cloud Data Hub (CDH). However, the initial version of CDH supported only coarse-grained access control to entire data assets, and hence it was not possible to scope access to data asset subsets.

Data Lake 109

Data Lake Sales Metadata Machine Learning 109

AWS Big Data

OCTOBER 30, 2024

Amazon DataZone now launched authentication supports through the Amazon Athena JDBC driver, allowing data users to seamlessly query their subscribed data lake assets via popular business intelligence (BI) and analytics tools like Tableau, Power BI, Excel, SQL Workbench, DBeaver, and more.

Visualization 119

Visualization Data Lake Testing Data Governance 119

DataKitchen

AUGUST 3, 2021

The data mesh design pattern breaks giant, monolithic enterprise data architectures into subsystems or domains, each managed by a dedicated team. A DataOps superstructure provides the foundation to address the many challenges inherent in operating a group of interdependent domains. The problem is not “you.”

Data Architecture 246

Data Architecture Data Lake Cost-Benefit Data Warehouse 246

Cloudera

NOVEMBER 13, 2024

We are excited to announce the acquisition of Octopai , a leading data lineage and catalog platform that provides data discovery and governance for enterprises to enhance their data-driven decision making.

Metadata 90

Metadata Management Data Governance Data-driven 90

DataKitchen

SEPTEMBER 17, 2021

Below is our final post (5 of 5) on combining data mesh with DataOps to foster innovation while addressing the challenges of a data mesh decentralized architecture. We see a DataOps process hub like the DataKitchen Platform playing a central supporting role in successfully implementing a data mesh.

Testing 237

Testing Data Architecture Measurement Visualization 237

DataKitchen

AUGUST 13, 2021

Figure 1 shows the 15-year cycle from screening to government agency approval and phase IV trials. If a company can use data to identify compounds more quickly and accelerate the development process, it can monetize its drug pipeline more effectively. How can they handle schema drift or data verification?

Testing 246

Testing Dashboards Marketing Measurement 246

datapine

MAY 31, 2022

2) The Challenges Of Cloud Computing. In this article, we have gathered the 12 most prominent challenges of cloud computing that will deliver fresh perspectives related to the market. These challenges of cloud computing are not merely roadblocks to overcome. Check out these 12 challenges and how to face them!

Risk 237

Risk Cost-Benefit Business Intelligence Data-driven 237

DataKitchen

NOVEMBER 18, 2021

For several years now, the elephant in the room has been that data and analytics projects are failing. Gartner estimated that 85% of big data projects fail. Add all these facts together, and it paints a picture that something is amiss in the data world. . The top-line result was that 97% of data engineers are feeling burnout. .

Testing 246

Testing Data Governance Measurement Software 246

DataKitchen

AUGUST 5, 2021

In our last post, we summarized the thinking behind the data mesh design pattern. In this post (2 of 5), we will review some of the ideas behind data mesh, take a functional look at data mesh and discuss some of the challenges of decentralized enterprise architectures like data mesh. Data Mesh Architecture Example.

Enterprise 246

Enterprise Management Data Processing Analytics 246

Cloudera

DECEMBER 17, 2024

Since 5G networks began rolling out commercially in 2019, telecom carriers have faced a wide range of new challenges: managing high-velocity workloads, reducing infrastructure costs, and adopting AI and automation. The challenge lies in finding a solution that enables growth while keeping infrastructure and operational expenses under control.

Enterprise 62

Enterprise Data Architecture Data-driven Optimization 62

erwin

OCTOBER 8, 2020

So if you’re going to move from your data from on-premise legacy data stores and warehouse systems to the cloud, you should do it right the first time. And as you make this transition, you need to understand what data you have, know where it is located, and govern it along the way. Then you must bulk load the legacy data.

Data Governance 145

Data Governance Metadata Testing Data Lake 145

erwin

JANUARY 30, 2020

I’m excited to share the results of our new study with Dataversity that examines how data governance attitudes and practices continue to evolve. Defining Data Governance: What Is Data Governance? . 1 reason to implement data governance. Most have only data governance operations.

Data Governance 145

Data Governance Metadata Cost-Benefit Digital Transformation 145

AWS Big Data

DECEMBER 4, 2024

Open table formats are emerging in the rapidly evolving domain of big data management, fundamentally altering the landscape of data storage and analysis. By providing a standardized framework for data representation, open table formats break down data silos, enhance data quality, and accelerate analytics at scale.

Snapshot 84

Snapshot Metadata Data Lake Optimization 84

datapine

SEPTEMBER 16, 2022

Previously, we discussed the top 19 big data books you need to read, followed by our rundown of the world’s top business intelligence books as well as our list of the best SQL books for beginners and intermediates. Data visualization, or ‘data viz’ as it’s commonly known, is the graphic presentation of data.

Visualization 304

Visualization Dashboards Data-driven Statistics 304

datapine

JULY 1, 2020

“The goal is to turn data into information, and information into insight.” – Carly Fiorina, former executive, president, HP. Digital data is all around us. quintillion bytes of data every single day, with 90% of the world’s digital insights generated in the last two years alone, according to Forbes.

Reporting 260

Reporting Dashboards Cost-Benefit Visualization 260

erwin

SEPTEMBER 10, 2020

Not Documenting End-to-End Data Lineage Is Risky Busines – Understanding your data’s origins is key to successful data governance. Not everyone understands what end-to-end data lineage is or why it is important. Data Lineage Tells an Important Origin Story. Who are the data owners?

Data Governance 131

Data Governance Key Performance Indicator Metadata Digital Transformation 131

DataKitchen

SEPTEMBER 20, 2021

Data organizations don’t always have the budget or schedule required for DataOps when conceived as a top-to-bottom, enterprise-wide transformational change. DataOps can and should be implemented in small steps that complement and build upon existing workflows and data pipelines. Figure 1: The four phases of Lean DataOps. production).

Testing 246

Testing Metrics Measurement Dashboards 246

erwin

FEBRUARY 6, 2020

As organizations deal with managing ever more data, the need to automate data management becomes clear. Last week erwin issued its 2020 State of Data Governance and Automation (DGA) Report. One piece of the research that stuck with me is that 70% of respondents spend 10 or more hours per week on data-related activities.

Management 144

Management Data Governance Cost-Benefit Metadata 144

DataRobot

JANUARY 20, 2022

The UK Government Health and Care Bill sets up Integrated Care Systems (ICSs) as legal entities from July 2022. Empowering ICSs to embrace healthcare transformation challenges as they bring together their component organizations, which will require significant clinical pathway and process redesign. Building data communities.

Data-driven 145

Data-driven Experimentation Predictive Modeling Data Warehouse 145

DataKitchen

OCTOBER 5, 2021

A DataOps Engineer owns the assembly line that’s used to build a data and analytic product. We find it helpful to think of data operations as a factory. We find it helpful to think of data operations as a factory. Most organizations run the data factory using manual labor. Figure 1: Ford assembly line, 1913.

Testing 154

Testing Dashboards Measurement Experimentation 154

datapine

SEPTEMBER 29, 2022

1) What Is Data Quality Management? 4) Data Quality Best Practices. 5) How Do You Measure Data Quality? 6) Data Quality Metrics Examples. 7) Data Quality Control: Use Case. 8) The Consequences Of Bad Data Quality. 9) 3 Sources Of Low-Quality Data. 10) Data Quality Solutions: Key Attributes.

Data Quality 241

Data Quality Metrics Data-driven Management 241

erwin

NOVEMBER 28, 2019

In the era of data-driven business, such perspective is critical. EA also enables a better understanding of change, or impact analysis – which is essential considering the agile, data-driven landscape and its state of flux. Roles and titles will continue to evolve to meet new challenges in the face of digital transformation.

Enterprise 145

Enterprise Data-driven Data Governance Metadata 145

Cloudera

OCTOBER 7, 2021

In this blog, I will demonstrate the value of Cloudera DataFlow (CDF) , the edge-to-cloud streaming data platform available on the Cloudera Data Platform (CDP) , as a Data integration and Democratization fabric. Introduction to the Data Mesh Architecture and its Required Capabilities. Introduction.

Metadata 124

Metadata Cost-Benefit Enterprise Interactive 124

erwin

JULY 22, 2020

Fostering organizational support for a data-driven culture might require a change in the organization’s culture. Recently, I co-hosted a webinar with our client E.ON , a global energy company that reinvented how it conducts business from branding to customer engagement – with data as the conduit. As an example, E.ON Avoiding Hurdles.

Data-driven 136

Data-driven Data Governance Digital Transformation Data Processing 136

O'Reilly on Data

MARCH 26, 2024

TL;DR LLMs and other GenAI models can reproduce significant chunks of training data. Specific prompts seem to “unlock” training data. Generative AI Has a Plagiarism Problem ChatGPT, for example, doesn’t memorize its training data, per se. So, is the training data “stored” in the model? Well, no, not quite. But also… Yes?

Modeling 317

Modeling Machine Learning Risk Advertising 317

DataKitchen

APRIL 28, 2021

Industry analysts who follow the data and analytics industry tell DataKitchen that they are receiving inquiries about “data fabrics” from enterprise clients on a near-daily basis. Gartner included data fabrics in their top ten trends for data and analytics in 2019. What is a Data Fabric?

Statistics 246

Statistics Optimization Data Analytics Technology 246

erwin

DECEMBER 6, 2019

In the data-driven era, CIO’s need a solid understanding of data governance 2.0 … Data governance (DG) is no longer about just compliance or relegated to the confines of IT. Today, data governance needs to be a ubiquitous part of your organization’s culture. Collaborative Data Governance.

Data Governance 145

Data Governance Metadata Enterprise Data-driven 145

TDAN

SEPTEMBER 1, 2020

Getting the business engaged with data governance can sometimes be a challenge. At NAIT (the Northern Alberta Institute of Technology), we have put together a process to visually identify and connect our reports to Data Governance. The […].

Data Governance 128

Data Governance Visualization Reporting Technology 128

erwin

SEPTEMBER 3, 2020

Why should you integrate data governance (DG) and enterprise architecture (EA)? Two of the biggest challenges in creating a successful enterprise architecture initiative are: collecting accurate information on application ecosystems and maintaining the information as application ecosystems change.

Data Governance 144

Data Governance Enterprise Risk Data Lake 144

erwin

MAY 20, 2020

Data governance (DG) as a an “emergency service” may be one critical lesson learned coming out of the COVID-19 crisis. Where crisis leads to vulnerability, data governance as an emergency service enables organization management to direct or redirect efforts to ensure activities continue and risks are mitigated.

Data Governance 144

Data Governance Metadata Risk Strategy 144

datapine

FEBRUARY 22, 2022

Over the past 5 years, big data and BI became more than just data science buzzwords. Without real-time insight into their data, businesses remain reactive, miss strategic growth opportunities, lose their competitive edge, fail to take advantage of cost savings options, don’t ensure customer satisfaction… the list goes on.

Business Intelligence 218

Business Intelligence Strategy Cost-Benefit Dashboards 218

DataKitchen

NOVEMBER 4, 2021

Data organizations often have a mix of centralized and decentralized activity. DataOps concerns itself with the complex flow of data across teams, data centers and organizational boundaries. It expands beyond tools and data architecture and views the data organization from the perspective of its processes and workflows.

Data Processing 246

Data Processing Data Lake Cost-Benefit Testing 246

Timo Elliott

JANUARY 6, 2022

Like the proverbial man looking for his keys under the streetlight , when it comes to enterprise data, if you only look at where the light is already shining, you can end up missing a lot. Remember that dark data is the data you have but don’t understand. So how do you find your dark data? Analyze your metadata.

IT 135

IT Metadata Data-driven Data Governance 135

DataKitchen

FEBRUARY 2, 2023

We’ve read many predictions for 2023 in the data field: they cover excellent topics like data mesh, observability, governance, lakehouses, LLMs, etc. What will the world of data tools be like at the end of 2025? Central IT Data Teams focus on standards, compliance, and cost reduction. Recession: the party is over.

Metadata 130

Metadata Testing Data Science Risk 130

AWS Big Data

JULY 29, 2024

In the era of big data, data lakes have emerged as a cornerstone for storing vast amounts of raw data in its native format. They support structured, semi-structured, and unstructured data, offering a flexible and scalable environment for data ingestion from multiple sources.

Metadata 126

Metadata Snapshot Data Lake Metrics 126

Cloudera

JULY 11, 2022

The need for data fabric. As Cloudera CMO David Moxey outlined in his blog , we live in a hybrid data world. Data is growing and continues to accelerate its growth. Driving insight and value from it all is as much of an opportunity as it is a challenge. Cloudera data fabric and analyst acclaim.

Data Architecture 107

Data Architecture Data-driven Data Warehouse Cost-Benefit 107