Exam Professional Machine Learning Engineer topic 1 question 90 discussion

In this scenario, the dataset is highly imbalanced, where most of the examples do not have the company's logo. Therefore, accuracy could be misleading as the model can have high accuracy by simply predicting that all images do not have the logo. F1 score is a good metric to consider in such cases, as it takes both precision and recall into account. However, since the dataset is highly skewed, we should weigh recall more than precision to ensure that the model is correctly identifying the images that do have the logo. Therefore, F-score where recall is weighed more than precision is the best metric to evaluate the performance of the model in this scenario. Option B (RMSE) is not applicable to this classification problem, and option D (F-score where precision is weighed more than recall) is not suitable for highly skewed datasets.

F1-score: Harmonic mean of precision and recall. It ensures that both fasle positive and false negatives are considerd. F1- focusing on recall may be useful if missing a logo is more costly than incorrectly identifying one.

I'd go with C. We don't know which option (less FP or less FN) is most important for business with the provided information, so we should seek a balance.

I think it's D.

I think it could be D, but the question does not provide enough information for this. I have this feeling: If 4% have the logo, we are looking just for these ones, right? So, the 'quality of TP,' that's it, the precision, could be more interesting because we want a model that we can rely on. So, when this model Predict a image with logo, we`ll be more certain about it. If we use recall, for example, a model with 99% recall has more chance of getting the logo, but we won't have quality in this. This model could suggest a lot of images without logo. It is better to use any ML than this...

both option A (F-score with higher weight on recall) and option C (F1 score) could be suitable depending on the specific priorities and requirements of your classification problem. If missing a company's logo is considered more problematic than having false alarms, then option A might be preferred. The F1 score (option C) is a balanced measure that considers both precision and recall, which is generally a good choice in imbalanced datasets. Ultimately, the choice between option A and option C depends on the specific goals and constraints of your application.

The question not have clear preference for recall or precision hence going with C

Yeah, I know - everyone is voting A... To be honest I still don't understand why are you more affraid of these few FNs than FPs. In my opinion they are exactly same evil. Every documantation says that F1 is great on skewed data. You should use weighted F1 when you know what is worse for you FNs or FPs. In this case we have no any hints on it, so I would stay with ordinary F1.

A. F-score where recall is weighed more than precision Even a model which always says that don't have the logo will have a good precision because is the most common. What we need is improve recall.

Went with A

I think is A. The positive Class is the minority. So, it's more important to correctly detect logos in all images that have logo (recall) than correctly detect logos in images classified with logos (precision).

I think is D becouse u try detect TP then it's more important recall than precision

Answer A is my choice.

A is correct

less logo images. Recall should be weighted more

I think A. If D were the answer, the threshold would be set higher to increase PRECISION, but the low percentage of positives (4%) would allow RECALL to be extremely low. If the percentage of positives is low, greater weight should be given to RECALL. https://medium.com/@douglaspsteen/beyond-the-f-1-score-a-look-at-the-f-beta-score-3743ac2ef6e3

Answer C: F1-Score is the best for imbalanced Data like this case: https://stephenallwright.com/imbalanced-data-metric/