AWS Big Data

APRIL 19, 2023

Independent data products often only have value if you can connect them, join them, and correlate them to create a higher order data product that creates additional insights. A modern data architecture is critical in order to become a data-driven organization.

Technology 93

Technology Data-driven Machine Learning Sales 93

Cloudera

JUNE 13, 2024

After countless open-source innovations ushered in the Big Data era, including the first commercial distribution of HDFS (Apache Hadoop Distributed File System), commonly referred to as Hadoop, the two companies joined forces, giving birth to an entire ecosystem of technology and tech companies.

Big Data 103

Big Data Machine Learning Contextual Data Data Lake 103

Sisense

AUGUST 28, 2019

Simply put, the term cloud-agnostic refers to the ability to move applications or parts of applications from one cloud platform to another. What does it mean for your data? Let’s dive into what you should consider in a BI platform to ensure you’re protecting and future-proofing your company’s data strategy.

Analytics 73

Analytics Data Strategy Strategy Data Architecture 73

AWS Big Data

SEPTEMBER 13, 2023

The Analytics specialty practice of AWS Professional Services (AWS ProServe) helps customers across the globe with modern data architecture implementations on the AWS Cloud. The company wanted the ability to continue processing operational data in the secondary Region in the rare event of primary Region failure.

Data Lake 104

Data Lake Data Processing Metadata Snapshot 104

Sisense

JULY 6, 2020

First off, this involves defining workflows for every business process within the enterprise: the what, how, why, who, when, and where aspects of data. These regulations, ultimately, ensure key business values: data consistency, quality, and trustworthiness.

Enterprise 98

Enterprise Management Data Architecture Data-driven 98

IBM Big Data Hub

FEBRUARY 9, 2023

Realize that a data governance program cannot exist on its own – it must solve business problems and deliver outcomes. Start by identifying business objectives, desired outcomes, key stakeholders, and the data needed to deliver these objectives. Don’t try to do everything at once!

Data Governance 76

Data Governance Business Objectives Data Quality Measurement 76

TDAN

OCTOBER 18, 2022

“Technical debt” refers to the implied cost of future refactoring or rework to improve the quality of an asset to make it easy to understand, work with, maintain, and extend.

Software 94

Software Data Architecture IT Data Quality 94

IBM Big Data Hub

SEPTEMBER 19, 2023

Though you may encounter the terms “data science” and “data analytics” being used interchangeably in conversations or online, they refer to two distinctly different concepts.

Data Science 76

Data Science Data Analytics Prescriptive Analytics Analytics 76

AWS Big Data

JANUARY 12, 2024

Amazon Kinesis and Amazon MSK also have capabilities to stream data directly to a data lake on Amazon S3. S3 data lake Using Amazon S3 for your data lake is in line with the modern data strategy. With this approach, you can bring compute to your data as needed and only pay for capacity it needs to run.

Data Lake 109

Data Lake Cost-Benefit Visualization Structured Data 109

TDAN

OCTOBER 1, 2019

Is there a difference between open data and public data? There is a general consensus that when we talk about open data, we are referring to any piece of data or content that is free to access, use, reuse, and redistribute. And it’s a big one. Due to the way most governments have rolled out […].

Data Architecture 80

Data Architecture Data Strategy Big Data Strategy 80

Peter James Thomas

MARCH 9, 2020

In the thirteen years that have passed since the beginning of 2007, I have helped ten organisations to develop commercially-focused Data Strategies [1]. However, in this initial article, I wanted to to focus on one tool that I have used as part of my Data Strategy engagements; a Data Maturity Model.

Measurement 67

Measurement Data Strategy Strategy Modeling 67

TDAN

NOVEMBER 19, 2019

Unless you’ve been living in a cave somewhere, you’ve certainly heard news about stolen and hacked data, commonly referred to as data breaches. They occur with great regularity, and there is no indication that the frequency of data breaches is slowing down.

Data Architecture 75

Data Architecture Data Strategy Strategy Management 75

Peter James Thomas

MAY 3, 2019

Control of Data to ensure it is Fit-for-Purpose. This refers to a wide range of activities from Data Governance to Data Management to Data Quality improvement and indeed related concepts such as Master Data Management. Data Architecture / Infrastructure. Data Strategy.

Strategy 55

Strategy Data Architecture Data Quality Data Strategy 55

AWS Big Data

APRIL 3, 2024

For instructions, refer to Amazon DataZone quickstart with AWS Glue data. You also need to define and run a ruleset against your data, which is a set of data quality rules in AWS Glue Data Quality. To learn more about Amazon DataZone, refer to the Amazon DataZone User Guide. option("header", "true").option("inferSchema",

Data Quality 104

Data Quality Visualization Metadata Metrics 104

AWS Big Data

APRIL 26, 2023

In the next stage of the cycle, Huron’s consultants experiment with new data sources and insights that in turn fed back into the product dashboards. This access to the metadata supports the need for updating visuals based on changes as well as automating row and column level security ensuring customer data is properly governed.

Metadata 52

Metadata Dashboards Visualization Consulting 52

AWS Big Data

JULY 16, 2024

When data is moved to the Infrequent Access tier, costs are reduced by up to 40%. Similarly, when data is moved to the Archive Instant Access tier, storage costs are reduced by up to 68%. Refer to Amazon S3 pricing for current pricing, as well as for information by region.

Insurance 124

Insurance Management Cost-Benefit Optimization 124

AWS Big Data

AUGUST 1, 2024

Success criteria alignment by all stakeholders (producers, consumers, operators, auditors) is key for successful transition to a new Amazon Redshift modern data architecture. The success criteria are the key performance indicators (KPIs) for each component of the data workflow.

Data Warehouse 100

Data Warehouse KPI Optimization Cost-Benefit 100

AWS Big Data

OCTOBER 25, 2023

To learn more about how to enable semantic search on unstructured data by integrating Amazon OpenSearch Service as a vector database, refer to Try semantic search with the Amazon OpenSearch Service vector engine. He is also the author of the book Simplify Big Data Analytics with Amazon EMR.

Unstructured Data 129

Unstructured Data Metadata Management Analytics 129

BizAcuity

APRIL 14, 2022

The comprehensive system which collectively includes generating data, storing the data, aggregating and analyzing the data, the tools, platforms and other softwares involved is referred to as Big Data Ecosystem. Data Management. Challenges associated with Data Management and Optimizing Big Data.

Big Data 52

Big Data Data Analytics Management Unstructured Data 52

Peter James Thomas

OCTOBER 30, 2018

Prelude… I recently came across an article in Marketing Week with the clickbait-worthy headline of Why the rise of the chief data officer will be short-lived (their choice of capitalisation). All the references I can find to it are modern pieces comparing it to the CDO role, so perhaps it is apochryphal.

Marketing 51

Marketing Strategy Data Architecture Data Strategy 51

IBM Big Data Hub

JANUARY 25, 2023

Breaking down these silos to encourage data access, data sharing and collaboration will be an important challenge for organizations in the coming years. The right data architecture to link and gain insight across silos requires the communication and coordination of a strategic data governance program.

Data Governance 52

Data Governance Data-driven Strategy Data Architecture 52

Alation

OCTOBER 12, 2021

Like many, the team at Cbus wanted to use data to more effectively drive the business. “Finding the right data was a real challenge,” recalls John Gilbert, Data Governance Manager. “There was no single source of reference, there was no catalog to leverage, and it was unclear who to ask or seek assistance from.”

Strategy 52

Strategy Enterprise Analytics Data Governance 52

BizAcuity

MAY 24, 2022

Without a well-thought of data strategy and framework in place, the cloud strategy could prove to be more expensive than it has to be. To try and test the platforms in accordance with data strategy and governance. Through every phase of cloud adoption, well, sky is still the limit. No pun intended.

Data-driven 52

Data-driven Cost-Benefit Digital Transformation Strategy 52

TDAN

MARCH 17, 2020

Earlier this month we hosted the second annual Data-Centric Architecture Forum (#DCAF2020) in Fort Collins, CO. Last year, (2019) we hosted the first Data-Centric Architecture conference. In 2019, the focus was on getting a sketch of a reference architecture (click here to see). These are the highlights.

Reporting 52

Reporting Data Processing Data Architecture Data Strategy 52

TDAN

JULY 19, 2022

In reference to the prior column on enterprise data management and high level lego framework, this column reviews in detail the foundational layer of Organization Mission, Level 1.

Strategy 52

Strategy Enterprise Management Modeling 52

TDAN

JANUARY 5, 2021

Most data is not static. No, data has a life in which it changes, is used for perhaps multiple purposes, and gets moved all over the place. So, it makes sense to think about the lifecycle of your data at your organization. The accompanying diagram helps to demonstrate this lifecycle.

IT 91

IT Data Architecture Data Strategy Strategy 91

Peter James Thomas

APRIL 3, 2020

I have been very much focussing on the start of a data journey in a series of recent articles about Data Strategy [3]. The way that this consistency of figures is achieved is by all elements of the Structured Reporting Framework drawing their data from the same data repositories. Introduction.

Dashboards 61

Dashboards Reporting Sales Data Lake 61

TDAN

AUGUST 20, 2019

We hope your Data Management career and programs are progressing well. If you have issues, please refer to DAMA.org for references, as well as the DAMA Data Management Body of Knowledge (DMBok). Good day from DAMA International. You can purchase the DMBoK at your favorite book source or via website link.

Management 40

Management Data Quality Data Architecture Data Governance 40

Peter James Thomas

OCTOBER 1, 2019

Note: Here an “axis” is a fixed reference line (sometimes invisible for stylistic reasons) which typically goes vertically up the page or horizontally from left to right across the page (but see also Radar Charts ). I use Radar Charts myself extensively when assessing organisations’ data capabilities. Scatter Charts.

Sales 61

Sales Measurement Statistics Data Architecture 61

Anmut

MARCH 24, 2022

This type of data landscape is usually created using innovative data discovery tools like Amundsen , Nemo , and DataHub as part of a company’s data transformation efforts. The use cases and customer outcomes your data supports and the quantifiable value your data creates for the business.

ROI 52

ROI Measurement Data-driven Data Transformation 52

AWS Big Data

MARCH 28, 2024

We also decided to use Amazon EMR managed scaling to scale the core and task nodes (for scaling scenarios, refer to Node allocation scenarios ). He is also the author of the book Simplify Big Data Analytics with Amazon EMR. Outside of work, Sakti enjoys learning new technologies, watching movies, and visiting places with family.

Optimization 93

Optimization IT Big Data Data Processing 93

AWS Big Data

OCTOBER 30, 2024

This is the final part of a three-part series where we show how to build a data lake on AWS using a modern data architecture. This post shows how to process data with Amazon Redshift Spectrum and create the gold (consumption) layer. His focus areas are MLOps, feature stores, data lakes, model hosting, and generative AI.

Data Lake 73

Data Lake Machine Learning Data Architecture Data-driven 73