Welcome to ExamTopics
ExamTopics Logo
- Expert Verified, Online, Free.
exam questions

Exam Professional Machine Learning Engineer All Questions

View all questions & answers for the Professional Machine Learning Engineer exam

Exam Professional Machine Learning Engineer topic 1 question 137 discussion

Actual exam question from Google's Professional Machine Learning Engineer
Question #: 137
Topic #: 1
[All Professional Machine Learning Engineer Questions]

You deployed an ML model into production a year ago. Every month, you collect all raw requests that were sent to your model prediction service during the previous month. You send a subset of these requests to a human labeling service to evaluate your model’s performance. After a year, you notice that your model's performance sometimes degrades significantly after a month, while other times it takes several months to notice any decrease in performance. The labeling service is costly, but you also need to avoid large performance degradations. You want to determine how often you should retrain your model to maintain a high level of performance while minimizing cost. What should you do?

  • A. Train an anomaly detection model on the training dataset, and run all incoming requests through this model. If an anomaly is detected, send the most recent serving data to the labeling service.
  • B. Identify temporal patterns in your model’s performance over the previous year. Based on these patterns, create a schedule for sending serving data to the labeling service for the next year.
  • C. Compare the cost of the labeling service with the lost revenue due to model performance degradation over the past year. If the lost revenue is greater than the cost of the labeling service, increase the frequency of model retraining; otherwise, decrease the model retraining frequency.
  • D. Run training-serving skew detection batch jobs every few days to compare the aggregate statistics of the features in the training dataset with recent serving data. If skew is detected, send the most recent serving data to the labeling service.
Show Suggested Answer Hide Answer
Suggested Answer: D 🗳️

Comments

Chosen Answer:
This is a voting comment (?) , you can switch to a simple comment.
Switch to a voting comment New
TNT87
Highly Voted 1 year, 8 months ago
Selected Answer: D
Option D is the best approach to determine how often to retrain the model while minimizing cost. Running training-serving skew detection batch jobs every few days to compare the aggregate statistics of the features in the training dataset with recent serving data is an effective way to detect when the model's performance has degraded. If skew is detected, the most recent serving data should be sent to the labeling service to evaluate the model's performance. This approach is more cost-effective than sending a subset of requests to the labeling service every month because it only sends data when there is a high probability that the model's performance has degraded. By doing this, the model can be retrained at the right time, and the cost of the labeling service can be minimized.
upvoted 5 times
...
f084277
Most Recent 1 week, 1 day ago
Selected Answer: D
Clearly D. B is just guesswork.
upvoted 1 times
...
M25
1 year, 6 months ago
Selected Answer: D
Went with D
upvoted 2 times
...
John_Pongthorn
1 year, 10 months ago
Selected Answer: D
D https://cloud.google.com/blog/topics/developers-practitioners/monitor-models-training-serving-skew-vertex-aiew-vertex-ai&ved=2ahUKEwiRg_aoj9n8AhWb7TgGHcGCDREQFnoECAwQAQ&usg=AOvVaw197NneIJM0ra7fLq2zsOin
upvoted 2 times
...
ares81
1 year, 10 months ago
Selected Answer: B
B looks the only option, to me.
upvoted 2 times
...
hiromi
1 year, 11 months ago
Selected Answer: D
D - https://cloud.google.com/blog/topics/developers-practitioners/monitor-models-training-serving-skew-vertex-ai - https://developers.google.com/machine-learning/guides/rules-of-ml
upvoted 4 times
...
mymy9418
1 year, 11 months ago
Selected Answer: D
I think D
upvoted 1 times
...
mil_spyro
1 year, 11 months ago
Selected Answer: B
"After a year, you notice that your model's performance sometimes degrades significantly after a month, while other times it takes several months to notice any decrease in performance." Hence I vote B
upvoted 2 times
...
Community vote distribution
A (35%)
C (25%)
B (20%)
Other
Most Voted
A voting comment increases the vote count for the chosen answer by one.

Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.

SaveCancel
Loading ...