Exam Professional Machine Learning Engineer topic 1 question 73 discussion

https://www.tensorflow.org/tfx/guide/evaluator

it's seem C for me B is wrong cuz "Many machine learning techniques don’t work well here due to the sequential nature and temporal correlation of time series. For example, k-fold cross validation can cause data leakage; models need to be retrained to generate new forecasts" - https://cloud.google.com/learn/what-is-time-series

A) TFX ModelValidator is designed to handle the exact needs described in the scenario: training on all data, validating on specific subsets, and ensuring production readiness with comprehensive performance metrics. This makes it the most streamlined and reliable method compared to other options, which either lack specificity in production readiness (B), are too narrow in scope (C), or risk overfitting and inadequate validation (D).

Evaluator TFX lets you evaluate the performance on different subsets of data https://www.tensorflow.org/tfx/guide/evaluator

The Evaluator TFX pipeline component performs deep analysis on the training results for your models, to help you understand how your model performs on subsets of your data.

I prefer A to C because 1 week of data may be insufficient to generalize the model and could lead to overfitting on the validation subset.

option C provides a streamlined and reliable approach that focuses on evaluating the model's performance on the most relevant and recent data, which is essential for predicting out-of-stock events in a dynamic retail setting.

Either A or C but C is only last week which is not specific data sets

Answer should be C, we are dealing with dynamic data and the "last" data is more relevant to have an idea about the future performance

Option C, because it allows you to track your model's performance on the most *recent* data, which is the most relevant data for predicting stockout risk. Given that the preferences are dynamic, the most important thing is that the model WORKS correctly with the newest data

The answer is A. Performance on specific subsets of data before pushing to production == TFX ModelValidator with custom performance metrics for production readiness. C is wrong because performance in the last relevant week of data != performance on specific subsets of data.

I will go for A. I don't think the aim of the question is to test if the candidates know whether or not a component is deprecated . Note that ModelValidator has been fused with Evaluator. So we can imagine, the question would have been updated in recent exams. Evaluator enables testing on specific subsets with the metrics we want, then indicates to Pusher component to push the new model to production if "model is good enough". This would make the pipeline quite streamlined (https://www.tensorflow.org/tfx/guide/evaluator) B: wrong: using historical data, one should watch data leakage C: wrong: We want to track performance on specific subsets of data (not necessarily the last week) maybe to do some targeting/segmentation ? who knows. D: wrong because we want to track performance on specific subsets of data not the entire dataset

Went with C

I think that it should be C for the following key point ", but track your performance on specific subsets of data before pushing to production" So the ask is which subset of data you should use.

Could someone explain why A is better option than C? C is correct one in terms of evaluation overall, no doubt. But do we choose TFX because it understands we are dealing with time series? Or is it the "specific subset" in the Q that makes us thinking we have already chosen the data of last period and just need to push it into the TFX?

A is deprecated.. so C

Went with A