Welcome to ExamTopics
ExamTopics Logo
- Expert Verified, Online, Free.
Mail Us[email protected]
Menu
Google Discussions
exam questions

Exam Professional Machine Learning Engineer All Questions

View all questions & answers for the Professional Machine Learning Engineer exam
Go to Exam

Exam Professional Machine Learning Engineer topic 1 question 102 discussion

Actual exam question from Google's Professional Machine Learning Engineer
Question #: 102
Topic #: 1
[All Professional Machine Learning Engineer Questions]

You have successfully deployed to production a large and complex TensorFlow model trained on tabular data. You want to predict the lifetime value (LTV) field for each subscription stored in the BigQuery table named subscription. subscriptionPurchase in the project named my-fortune500-company-project.

You have organized all your training code, from preprocessing data from the BigQuery table up to deploying the validated model to the Vertex AI endpoint, into a TensorFlow Extended (TFX) pipeline. You want to prevent prediction drift, i.e., a situation when a feature data distribution in production changes significantly over time. What should you do?

  • A. Implement continuous retraining of the model daily using Vertex AI Pipelines.
  • B. Add a model monitoring job where 10% of incoming predictions are sampled 24 hours.
  • C. Add a model monitoring job where 90% of incoming predictions are sampled 24 hours.
  • D. Add a model monitoring job where 10% of incoming predictions are sampled every hour.
Show Suggested Answer Hide Answer
Suggested Answer: B 🗳️
by hiromi at Dec. 20, 2022, 11:36 a.m.

Comments

Chosen Answer:
This is a voting comment (?) , you can switch to a simple comment.
Switch to a voting comment New
Submit Cancel
f084277
1 week, 2 days ago
Selected Answer: A
It says PREVENT with no other constraints.
upvoted 1 times
...
MultiCloudIronMan
7 months, 3 weeks ago
Selected Answer: B
You need to monitor it first and foremost to see if there is a drift and if there is then a measure can be devised. training every date is an over kill.
upvoted 2 times
...
pico
1 year, 1 month ago
Selected Answer: A
Continuous Retraining: Continuously retraining the model allows it to adapt to changes in the data distribution, helping to mitigate prediction drift. Daily retraining provides a good balance between staying up-to-date and avoiding excessive retraining. Options B, C, and D involve model monitoring but do not address the issue of keeping the model updated with the changing data distribution. Monitoring alone can help you detect drift, but it does not actively prevent it. Retraining the model is necessary to address drift effectively.
upvoted 3 times
maukaba
1 year, 1 month ago
Option A can prevent drift prediction. All the other options can only detect. Therefore the correct answer is A unless it is possible to monitor drifts and then remediate without retrainings.
upvoted 1 times
...
Nish1729
10 months, 3 weeks ago
Follow me on X (twitter): @nbcodes for more useful tips. I think you're slightly missing the point, the answer should be B, let me explain why.. The whole point of this question is to come up with a PREVENTATIVE way of handling prediction drift so you need to find a way to DETECT the drift before it occurs, this is exactly what solution B does and ensures it's done in a way that is not too frequent i.e D and not too resource intensive with the large sample i.e C remember if sampling is done well you don't need 90% of the data to detect drift. Solution A suggests retraining every day which is a CRAZY proposal, why would you retrain every day even if you don't know if your data is drifting?? Huge waste of resources and time.
upvoted 2 times
...
...
M25
1 year, 6 months ago
Selected Answer: B
Went with B
upvoted 1 times
...
tavva_prudhvi
1 year, 8 months ago
Selected Answer: B
Continuous retraining (option A) is not necessarily the best solution for preventing prediction drift, as it can be time-consuming and expensive. Instead, monitoring the performance of the model in production is a better approach. Option B is a good choice because it samples a small percentage of incoming predictions and checks for any significant changes in the feature data distribution over a 24-hour period. This allows you to detect any drift and take appropriate action to address it before it affects the model's performance. Options C and D are less effective because they either sample too many or too few predictions and/or at too frequent intervals.
upvoted 4 times
andresvelasco
1 year, 2 months ago
I am just not sure why sampling too few (10%) is important. Is this a costly service?
upvoted 1 times
tavva_prudhvi
1 year ago
Model monitoring, especially at a large scale, can consume significant computational resources. Sampling a smaller percentage of predictions (like 10%) helps manage these resource demands and associated costs. The more predictions you sample, the more storage, computation, and network resources you'll need to analyze the data, potentially increasing the cost. In many cases, a 10% sample of the data can provide statistically significant insights into the model's performance and the presence of drift. It's a balancing act between getting enough data to make informed decisions and not overburdening the system. In some datasets, especially large ones, a lot of the data might be redundant or not particularly informative. Sampling a smaller fraction can help filter out noise and focus on the most relevant information.
upvoted 1 times
...
...
pico
1 year ago
Neither B,C or D have a step to prevent the prediction drift. The question says: "you want to prevent prediction drift"
upvoted 1 times
...
...
TNT87
1 year, 8 months ago
Selected Answer: B
Answer B
upvoted 1 times
...
John_Pongthorn
1 year, 9 months ago
Selected Answer: B
B , I got it from Machine Learning in the Enterprise course for google partnet skillboost you can watch cafully on video "Model management using Vertex AI" I imply that it is default setting on typical case.
upvoted 3 times
...
behzadsw
1 year, 10 months ago
Selected Answer: D
Using 10% of hourly requests would yield a better distribution and faster feed back loop
upvoted 1 times
...
hargur
1 year, 11 months ago
I think it is B, we can say 10% to be a sample but not 90%
upvoted 2 times
...
mymy9418
1 year, 11 months ago
Selected Answer: B
I guess 10% of 24 hours should be good enough?
upvoted 3 times
...
hiromi
1 year, 11 months ago
Selected Answer: B
B (not sure) - https://cloud.google.com/vertex-ai/docs/model-monitoring/overview - https://cloud.google.com/vertex-ai/docs/model-monitoring/using-model-monitoring#drift-detection
upvoted 2 times
...

Get IT Certification

Unlock free, top-quality video courses on ExamTopics with a simple registration. Elevate your learning journey with our expertly curated content. Register now to access a diverse range of educational resources designed for your success. Start learning today with ExamTopics!

Start Learning for free
Community vote distribution
A (35%)
C (25%)
B (20%)
Other
Most Voted
A voting comment increases the vote count for the chosen answer by one.

Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.

SaveCancel