Exam Professional Machine Learning Engineer topic 1 question 21 discussion

Answer is D

D is correct. Choose the feature / capability GCP provides is always a good bet. :)

[B] Compare the loss performance for each model on the validation data. --> Not validation data but testing data

The answer is A. I am not sure why people choose B vs A as you may overfit your validation set. And you are using your held-out set really rare == no option to overfit.

Continuous Evaluation feature is deprecated.

In the official study guide, this was the explanation given for answer B : "The image classification model is a deep learning model. You minimize the loss of deep learning models to get the best model. So comparing loss performance for each model on validation data is the correct answer."

D - because you are using a Google provided feature. remember in this exam its important to always choose the google services over anything else

mAP is for object detection, so the answer should be B

Went with D, using continuous evaluation feature seems correct to me.

I choose by myself D. But as I read the post here https://www.v7labs.com/blog/mean-average-precision, I was not sure about D. It wrote mAP is commonly used for object detection or instance segmentation tasks. Validation Dataset in GCP context: not trained dataset and not seen dataset

D. Compare the mean average precision across the models using the Continuous Evaluation feature https://cloud.google.com/vertex-ai/docs/evaluation/introduction Vertex AI provides model evaluation metrics, such as precision and recall, to help you determine the performance of your models... Vertex AI supports evaluation of the following model types: AuPRC: The area under the precision-recall (PR) curve, also referred to as average precision. This value ranges from zero to one, where a higher value indicates a higher-quality model.

Went with D

I go for B. Option D is good when we are already in production

o monitor the performance of the model versions over time, you should compare the loss performance for each model on the validation data. Therefore, option B is the correct answer.

The best option to monitor the performance of multiple versions of an image classification model on AI Platform over time is to compare the loss performance for each model on the validation data. Option B is the best approach because comparing the loss performance of each model on the validation data is a common method to monitor machine learning model performance over time. The validation data is a subset of the data that is not used for model training, but is used to evaluate its performance during training and to compare different versions of the model. By comparing the loss performance of each model on the same validation data, you can determine which version of the model has better performance.

If you have multiple model versions in a single model and have created an evaluation job for each one, you can view a chart comparing the mean average precision of the model versions over time

Guys, I not sure about the answer D ... And maybe you could help me in my arguments. I think choose loss to compare the model performance is better than see for metrics. For example, when can build an image model classification that has good precision metrics, because the class in unbalanced, but the loss could be terrible because of kind of loss choose that penalizes classes. so, losses are better than metrics to available models, and the answer is in A or B. I thought that the A could be the answer because I see validation as a part of the training process. So, If we want to test the model performance over time, we have to use new data, which I suppose to be the held-out data.