Exam Professional Machine Learning Engineer All Questions

View all questions & answers for the Professional Machine Learning Engineer exam

Exam Professional Machine Learning Engineer topic 1 question 141 discussion

Actual exam question from Google's Professional Machine Learning Engineer

Question #: 141
Topic #: 1

[All Professional Machine Learning Engineer Questions]

You work for a magazine publisher and have been tasked with predicting whether customers will cancel their annual subscription. In your exploratory data analysis, you find that 90% of individuals renew their subscription every year, and only 10% of individuals cancel their subscription. After training a NN Classifier, your model predicts those who cancel their subscription with 99% accuracy and predicts those who renew their subscription with 82% accuracy. How should you interpret these results?

A. This is not a good result because the model should have a higher accuracy for those who renew their subscription than for those who cancel their subscription.
B. This is not a good result because the model is performing worse than predicting that people will always renew their subscription.
C. This is a good result because predicting those who cancel their subscription is more difficult, since there is less data for this group.
D. This is a good result because the accuracy across both groups is greater than 80%.

Show Suggested Answer

Suggested Answer: C 🗳️

by TNT87 at Dec. 27, 2022, 10:06 a.m.

Comments

Submit Cancel

TNT87

Highly Voted 2 years, 6 months ago

Selected Answer: C

Answer C

upvoted 7 times

...

4d742d7

Most Recent 1 month, 1 week ago

Selected Answer: B

In imbalanced classification problems (like this one, where 90% renew and 10% cancel), accuracy alone is misleading. Here's why: Let's break it down: Suppose you always predict "renew" (i.e., the majority class). You’d be right 90% of the time. That’s already 90% accuracy with no model at all.

upvoted 1 times

...

desertlotus1211

4 months, 2 weeks ago

Selected Answer: B

Because 90% of subscribers renew, a simple baseline that always predicts "renew" would achieve 90% accuracy overall. In your model, while the cancellation (minority class) accuracy is high at 99%, the accuracy for renewals (the majority class) is only 82%. This means that when predicting the renewals, the model is doing significantly worse than the baseline of always predicting renewal.

upvoted 1 times

...

Amer95

4 months, 2 weeks ago

Selected Answer: B

Correct Answer: This is not a good result because the model is performing worse than a simple heuristic of predicting that everyone will renew. Why? • A simple heuristic (predicting all renewals) achieves 90% accuracy. • The model’s 82% accuracy for renewals is worse than this baseline, meaning it adds no real value. • The 99% accuracy for cancellations is misleading, likely due to severe class imbalance. • Alternative metrics (precision, recall, F1-score) would provide a clearer picture of actual performance.

upvoted 1 times

...

phani49

7 months ago

Selected Answer: B

Option B: "This is not a good result because the model is performing worse than predicting that people will always renew their subscription." Note: In practice, you'd also consider precision, recall, and business objectives. For example, if your business goal is to identify and retain canceling customers before they leave, a model with higher recall for cancellations might be beneficial despite the lower overall accuracy. But within the context of the given multiple-choice answers and the question's framing, B is the correct interpretation.

upvoted 1 times

...

lunalongo

7 months, 1 week ago

Selected Answer: B

B) - Saying "good result" is premature: no precision, recall, F1 was given - High accuracy over the minority could be overfitting - If the model predicted always renewal, it would achieve 90% accuracy. - 82% accuracy for renewals shows it isn't much better than a naive prediction.

upvoted 1 times

...

f084277

8 months ago

Selected Answer: C

C. The goal of the model is to predict CANCELLATIONS, not renewals

upvoted 1 times

...

pico

1 year, 8 months ago

Selected Answer: B

Here's the reasoning: The overall renewal rate is 90%, meaning that if the model simply predicted that everyone would renew, it would have an accuracy of 90%. The model's accuracy for predicting renewals (82%) is lower than this baseline accuracy. The model's accuracy for predicting cancellations is high (99%), but this could be misleading. If only 10% of individuals cancel their subscription, a model that predicts no cancellations at all would still have a high accuracy of 90%. Therefore, the high accuracy for cancellations may not be very informative. In summary, the model is not performing well, especially when compared to a simple baseline of always predicting renewals.

upvoted 3 times

ccb23cc

1 year, 1 month ago

if we suppose the case where the model simply predicted that everyone would renew, the renewals rate should be always higher than the cancellations. Therefore, This case means that the model made some asumptions about how a cancellation looks like and misled some of the renewals cases (it could make some wrong asumptions because there are few data)

upvoted 1 times

...

pico

1 year, 8 months ago

C suggests that predicting cancellations is more difficult due to less data for this group. While it's true that imbalanced datasets, where one class is underrepresented, can pose challenges for machine learning models, the key issue here is that the model's accuracy for predicting renewals is lower than the accuracy for predicting cancellations. In this scenario, the imbalance alone does not explain the lower accuracy for renewals. The model should ideally perform well on both classes, and the fact that it doesn't, especially when compared to a simple baseline of always predicting renewals (which would have an accuracy of 90%), suggests that there's a problem with the model's performance. Therefore, option B is a more appropriate interpretation, highlighting that the model is performing worse than a basic strategy of always predicting renewals.

upvoted 2 times

...

tavva_prudhvi

2 years ago

Selected Answer: C

since there is less data for this group. While the accuracy for predicting subscription renewals is lower, it is still above chance and may still be useful. Additionally, the high accuracy for predicting cancellations is promising, as this is the group of interest for the publisher. However, it would still be important to assess the model's precision and recall to fully evaluate its performance.

upvoted 2 times

...

Voyager2

2 years, 1 month ago

Selected Answer: C

Went with C: This is a good result because predicting those who cancel their subscription is more difficult, since there is less data for this group My Reason: "You have been tasked with predicting whether customers will cancel their annual subscription." And in that task you are getting 99% of accuracy

upvoted 4 times

...

ShePiDai

2 years, 1 month ago

Selected Answer: B

Task is to predict whether customer will cancel subscription, so both renew and cancel predictions are important. The overall accuracy is 99% x 10% + 82% x 90% = 83%, while guessing always renew has 90% accuracy.

upvoted 2 times

...

M25

2 years, 2 months ago

Selected Answer: B

#ResponsibleAI, predicting the majority class (imbalanced data) topic: “the model [82% accuracy for renew] is performing worse than predicting that people will always [90% accuracy] renew their subscription”. https://developers.google.com/machine-learning/crash-course/classification/check-your-understanding-accuracy-precision-recall “A deadly, but curable, medical condition afflicts .01% of the population. An ML model (…) predicts (…) with an accuracy of 99.99%. (…) After all, even a "dumb" model that always predicts "not sick" would still be 99.99% accurate.“

upvoted 2 times

...

lucaluca1982

2 years, 2 months ago

Selected Answer: B

The 82% accuracy for renewals is lower than a naive model that always predicts renewals (which would have a 90% accuracy).

upvoted 1 times

f084277

8 months ago

The goal of the model is to predict cancellations, not renewals...

upvoted 1 times

...

Scipione_

2 years, 5 months ago

Selected Answer: C

I think C is the only way

upvoted 3 times

...

John_Pongthorn

2 years, 5 months ago

Selected Answer: C

We can consider it as follows reasonably. A: it doesn't make any sense, given that cancel=99% but renew =82%, how did you make renew class (82%) beat the Cancel class(99%), it must be 100% accuracy ( bullshit) B: Cancel class have more accuracy than renew (99%>82%) D: You can justify, both are good 80% if we have a balance class. So it left us with C. This model predicts well upon imbalanced class circumstances. target class =10 samles meanwhile the another =90 samples

upvoted 2 times

...

ares81

2 years, 6 months ago

Selected Answer: C

Logically, it should be C.

upvoted 2 times

...

Dataspire

2 years, 6 months ago

Selected Answer: A

Since 90% of dataset represent customer who will renew subscription, accuracy should have been greater than 82%

upvoted 1 times

John_Pongthorn

2 years, 5 months ago

I think C is the most likely. we are experiencing an imbalanced dataset and the target class is canceled. Actually this case we have to use other metrics like F1 and precision/recall If you want to get ReNew accuracy higher than CAncel, you have to make it greater than 99% as compared to another. it is hard to archive.

upvoted 1 times

...