O'Reilly on Data

OCTOBER 13, 2020

In Bringing an AI Product to Market , we distinguished the debugging phase of product development from pre-deployment evaluation and testing. During testing and evaluation, application performance is important, but not critical to success. require not only disclosure, but also monitored testing. Debugging AI Products.

Management 314

Management Machine Learning Metrics Modeling 314

DataKitchen

OCTOBER 5, 2021

DataOps enables: Rapid experimentation and innovation for the fastest delivery of new insights to customers. In this blog, we’ll explore the role of the DataOps Engineer in driving the data organization to higher levels of productivity. A more technical discussion will follow in the next edition of this blog series.

Testing 152

Testing Dashboards Measurement Experimentation 152

Aryng

APRIL 13, 2022

“Experimentation is the least arrogant method of gaining knowledge. The experimenter humbly asks a question of nature.” For companies […] The post How to use Experimentation as a Growth Accelerator appeared first on Aryng's Blog.

Experimentation 52

Experimentation IT Testing 52

Domino Data Lab

MAY 15, 2019

While the talk provides both organizational foundations for machine learning as well as product management insights to consider when shipping ML projects, I will be focusing on the latter in this blog post. These steps also reflect the experimental nature of ML product management. more probabilistic rather than deterministic).

Machine Learning 111

Machine Learning Management Experimentation Uncertainty 111

The Unofficial Google Data Science Blog

APRIL 23, 2024

This blog post discusses such a comprehensive approach that is used at Youtube. To find optimal values of two parameters experimentally, the obvious strategy would be to experiment with and update them in separate, sequential stages. And we can keep repeating this approach, relying on intuition and luck.

Experimentation 52

Experimentation Optimization Uncertainty Metrics 52

Aryng

APRIL 22, 2022

Most managers are good at formulating innovative […] The post How to differentiate the thin line separating innovation and risk in experimentation appeared first on Aryng's Blog. We have seen this as a general trend in start-ups, and we know that it’s an awful feeling!

Experimentation 52

Experimentation Risk Management IT 52

datapine

FEBRUARY 4, 2020

As we have already talked about in our previous blog post on sales reports for daily, weekly or monthly reporting, you need to figure out a couple of things when launching and executing a marketing campaign: are your efforts paying off? 1) Blog Traffic And Blog Leads Report. click to enlarge**.

Reporting 280

Reporting Marketing Advertising Metrics 280

The Unofficial Google Data Science Blog

JULY 22, 2020

Another reason to use ramp-up is to test if a website's infrastructure can handle deploying a new arm to all of its users. The website wants to make sure they have the infrastructure to handle the feature while testing if engagement increases enough to justify the infrastructure. We offer two examples where this may be the case.

Experimentation 96

Experimentation Statistics Testing Knowledge Discovery 96

Smart Data Collective

JUNE 29, 2022

It could be a helpful blog post, an insightful whitepaper, or just a quick tip. Test Different Calls-to-Action. You will need to test different CTAs, which is going to require data analytics tools. Test Different Professional Email Signature. You should also use data analytics to test different email signatures.

B2B 124

B2B Data-driven Marketing Strategy 124

datapine

JANUARY 31, 2022

Everything is being tested, and then the campaigns that succeed get more money put into them, while the others aren’t repeated. This methodology of “test, look at the data, adjust” is at the heart and soul of business intelligence. Your Chance: Want to try a professional BI analytics software?

Business Intelligence 214

Business Intelligence Analytics Cost-Benefit ROI 214

DataRobot Blog

JANUARY 10, 2023

ML model builders spend a ton of time running multiple experiments in a data science notebook environment before moving the well-tested and robust models from those experiments to a secure, production-grade environment for general consumption. Capabilities Beyond Classic Jupyter for End-to-end Experimentation. Auto-scale compute.

Experimentation 52

Experimentation Machine Learning Data Science Modeling 52

The Unofficial Google Data Science Blog

JANUARY 16, 2018

We present data from Google Cloud Platform (GCP) as an example of how we use A/B testing when users are connected. Experimentation on networks A/B testing is a standard method of measuring the effect of changes by randomizing samples into different treatment groups. This simulation is based on the actual user network of GCP.

Testing 59

Testing Experimentation Measurement Modeling 59

O'Reilly on Data

MARCH 23, 2023

It’s by far the most convincing example of a conversation with a machine; it has certainly passed the Turing test. Search and research Microsoft is currently beta testing Bing/Sydney, which is based on GPT-4. There’s the compute time, the engineering team—but there’s also the cost of verification, testing, and editing.

IT 348

IT Modeling Testing Risk 348

Occam's Razor

MAY 15, 2006

There are many types of qualitative data at your disposal including brand buzz, customer satisfaction, net promoter indices, visitor engagement, stickiness, blog-pulse, etc. There is a lot of "buzz" around "buzzy" metrics such as brand value / brand impact, blog-pulse , to name a couple.

Metrics 100

Metrics Experimentation Testing Interactive 100

Cloudera

SEPTEMBER 18, 2024

Unique Data Integration and Experimentation Capabilities: Enable users to bridge the gap between choosing from and experimenting with several data sources and testing multiple AI foundational models, enabling quicker iterations and more effective testing.

Data-driven 91

Data-driven Optimization Experimentation Testing 91

Ontotext

JANUARY 10, 2025

This blog post summarizes our findings, focusing on NER as a first-step key task for knowledge extraction. Our goal is to test whether GenAI can handle diverse domains effectively and determine if its a viable tool for domain-specific graph-building tasks.

Informatics 59

Informatics Modeling Metadata Experimentation 59

Data Science 101

SEPTEMBER 1, 2023

I remember helping with my school website and also working on several WordPress blogs for myself and friends/family. If I had more room for experimentation though, I’d definitely give svelte and solidjs a try. I’ve been tinkering around with the web since I started learning how to code way back in high school.

Experimentation 116

Experimentation Testing Technology IT 116

CIO Business Intelligence

JULY 11, 2023

In this blog post, we will delve deeper into each of these principles and provide concrete examples to illustrate their importance. For example, a good result in a single clinical trial may be enough to consider an experimental treatment or follow-on trial but not enough to change the standard of care for all patients with a specific disease.

Modeling 95

Modeling Testing Experimentation Risk Management 95

Domino Data Lab

OCTOBER 14, 2020

Experimentation and collaboration are built into the core of the platform. This ability enhances the efficiency of operational management and optimizes the cost of experimentation. For a complete understanding of the benefits of Petastorm, check out this blog. Domino shines in reproducibility and discovery. Why Petastorm?

Experimentation 99

Experimentation Deep Learning Modeling Testing 99

DataRobot

JANUARY 20, 2022

— Collaborating via Snowflake Data Cloud and DataRobot AI Cloud Platform will enable multiple organizations to build a community movement where experimentation, innovation, and creativity flourish. As ICSs mature digitally, there is a need to ensure that all processes, datasets, and models are transparent and are free from bias. –

Data-driven 145

Data-driven Experimentation Predictive Modeling Data Warehouse 145

Decision Management Solutions

MARCH 26, 2020

We’ve been blogging recently on Decision Optimization. Randomly select groups of customers and use the experimental approach on them, to prevent bias, and ensure a clean test Keep information on both groups – what you would normally do and what you experimented on – so you can compare the approaches later.

Experimentation 72

Experimentation Recreation/Entertainment Optimization Testing 72

DataRobot Blog

MARCH 16, 2023

This means they need the tools that can help with testing and documenting the model, automation across the entire pipeline and they need to be able to seamlessly integrate the model into business critical applications or workflows. blog series and deep dive into the new 9.0 features over the next few weeks.

Experimentation 98

Experimentation Data-driven Modeling Metadata 98

Domino Data Lab

JUNE 13, 2019

Recently, Chhavi Yadav (NYU) and Leon Bottou (Facebook AI Research and NYU) indicated in their paper, “ Cold Case: The Lost MNIST Digits ”, how they reconstructed the MNIST (Modified National Institute of Standards and Technology) dataset and added 50,000 samples to the test set for a total of 60,000 samples. Did they overfit the test set?

Testing 83

Testing Data Science Experimentation Metadata 83

Sisense

JANUARY 21, 2020

A new drug promising to reduce the risk of heart attack was tested with two groups. Continuing the previous example, let’s assume that blood pressure is known to be a cause for heart attack and the goal of the test drug is to reduce blood pressure. It really depends on the circumstances. Combined 13/60 = 21.67% 11/60 = 18.3%.

Testing 104

Testing Data-driven Risk Statistics 104

The Unofficial Google Data Science Blog

OCTOBER 7, 2015

A/B testing is used widely in information technology companies to guide product development and improvements. For questions as disparate as website design and UI, prediction algorithms, or user flows within apps, live traffic tests help developers understand what works well for users and the business, and what doesn’t.

Modeling 63

Modeling Experimentation Knowledge Discovery Testing 63

IBM Big Data Hub

JUNE 10, 2024

In this blog, we’ll explore how businesses can use both on-premises and cloud XaaS to control budgets in the age of AI, driving financial sustainability without compromising on technological advancement. Embracing a culture of experimentation helps businesses drive innovation while minimizing financial risk.

Experimentation 62

Experimentation Optimization Risk Management Machine Learning 62

Cloudera

APRIL 9, 2021

This is part 4 in this blog series. This blog series follows the manufacturing and operations data lifecycle stages of an electric car manufacturer – typically experienced in large, data-driven manufacturing companies. The second blog dealt with creating and managing Data Enrichment pipelines. A/B testing).

Machine Learning 98

Machine Learning Forecasting Manufacturing Predictive Analytics 98

Octopai

MAY 2, 2022

Common elements of DataOps strategies include: Collaboration between data managers, developers and consumers A development environment conducive to experimentation Rapid deployment and iteration Automated testing Very low error rates. Issue detected? This means that more focused development can happen faster. .

Manufacturing 98

Manufacturing Testing Data Quality Dashboards 98

Cloudera

OCTOBER 18, 2023

In fact, some of the insights presented in this blog have been assisted by the power of large language models (LLMs), highlighting the synergy between human expertise and AI-driven insights. Testing: Generative AI can be used to generate test cases automatically, reducing the time and effort required for developers to test their code.

Cost-Benefit 79

Cost-Benefit Testing Machine Learning Optimization 79

Dataiku

MARCH 25, 2024

In this blog post, we delve into the workings of M-LLMs, unraveling the intricacies of their architecture, with a particular focus on text and vision integration. One limitation observed while testing the LENS approach, particularly in VQA, is its heavy reliance on the output of the first modules, namely CLIP and BLIP captions.

Visualization 132

Visualization Modeling Experimentation Testing 132

Domino Data Lab

APRIL 7, 2019

If you are in research, excellent libraries like Allen NLP and NLP Architect are designed to make experimentation easier, although at the expense of feature completeness, speed and robustness. Image Credit: Parsa Ghaffari on the Raylien Blog. These can be either classical or deep-learning-based pipelines.

Deep Learning 110

Deep Learning Machine Learning Experimentation Modeling 110

Occam's Razor

NOVEMBER 2, 2015

Higher Order Bits: Human vs. Business, Success KPIs, S-T-D-C Framework, MoR Test. If you are a regular reader of this blog, you know that perhaps the single greatest thing ever created here, :), is a simple framework you can apply to your entire digital strategy. It is pronounced the more test. Facebook for Businesses.

B2B 167

B2B Metrics Recreation/Entertainment Testing 167

Occam's Razor

FEBRUARY 26, 2008

" Or " I proposed testing / surveys / competitive intelligence / Analysts but I was shot down." 1: Implement a Experimentation & Testing Program. # 1: Implement a Experimentation & Testing Program. Here is data from our latest test." And now you have no excuse to avoid testing.

Management 68

Management Experimentation Testing Metrics 68

Depict Data Studio

JULY 13, 2021

a school district’s worth of students), it’s still unlikely you need statistics, unless you are trying to answer a scientific-type question (and what scientific-type questions nonprofits with a lot of data might ask is for another blog post on another day). Even if you have more people than that (e.g., Ask us more about Data Teams!

Statistics 98

Statistics Testing Recreation/Entertainment Experimentation 98

Cloudera

JANUARY 30, 2019

Data scientists require on-demand access to data, powerful processing infrastructure, and multiple tools and libraries for development and experimentation. Run experiments with historical reference for hyperparameter tuning, feature engineering, grid searches, A/B testing and more. Sound familiar?

Data Science 76

Data Science Machine Learning Experimentation Visualization 76

Occam's Razor

NOVEMBER 13, 2009

I now have several full time jobs, plus this blog, plus speaking around the world, plus a family, plus… so much more. " This blog post is in three parts: The pitch. Chapter 7 Failing Faster: Unleashing the Power of Testing and Experimentation. Sure you've heard of A/B and multivariate testing.

Analytics 71

Analytics Metrics Measurement Testing 71

Alation

DECEMBER 20, 2022

These organizations use experimentation and constant market testing to learn and invest. Subscribe to Alation's Blog. The change can fail and reveal insights into the organization that leads to better future transformation.” Parting Words. Successful digital transformation requires more than avoiding the above pitfalls.

Digital Transformation 96

Digital Transformation Scorecard Measurement Experimentation 96