Exam AWS Certified Machine Learning - Specialty topic 1 question 123 discussion

B and D

Compensate for imbalance and optimize on AUC. This is a class imbalance problem, not an overfitting problem.

the question didnt show the model performance on training data, so the overfitting issues is not correct.

A. NO - that will not address FN specifically but also FP B. YES - changing weight is best practice for class imbalance C. NO - there is no underfitting at 99.1% accuracy D. YES - AUC will address recall, which takes into account FN rate E. NO - there is no overfitting at 99.1% accuracy

Step B: Increasing the XGBoost scale_pos_weight parameter to adjust the balance of positive and negative weights can help the model deal with the imbalanced dataset. According to the XGBoost documentation, this parameter controls the balance of positive and negative weights, and is useful for unbalanced classes. A typical value to consider is sum(negative instances) / sum(positive instances). In this case, since there are 100 times more non-fraudulent transactions than fraudulent ones, setting scale_pos_weight to 100 can make the model more sensitive to the minority class and reduce false negatives. Step D: Changing the XGBoost eval_metric parameter to optimize based on Area Under the ROC Curve (AUC) can help the model focus on improving the true positive rate and the true negative rate, which are both important for fraud detection. According to the XGBoost

Step B: Increasing the XGBoost scale_pos_weight parameter to adjust the balance of positive and negative weights can help the model deal with the imbalanced dataset. According to the XGBoost documentation, this parameter controls the balance of positive and negative weights, and is useful for unbalanced classes. A typical value to consider is sum(negative instances) / sum(positive instances). In this case, since there are 100 times more non-fraudulent transactions than fraudulent ones, setting scale_pos_weight to 100 can make the model more sensitive to the minority class and reduce false negatives. Step D: Changing the XGBoost eval_metric parameter to optimize based on Area Under the ROC Curve (AUC) can help the model focus on improving the true positive rate and the true negative rate, which are both important for fraud detection.

I have some doubts about D and E. Precision-Recall AUC is better than AUC curve in imbalanced classes. Then, I choose E

Option A and Option E are unlikely to help reduce false negatives. Option C, increasing max_depth, may lead to overfitting, which could make the model worse. Option D, changing the eval_metric to optimize based on AUC, could help improve the model's ability to discriminate between the two classes. Option B, increasing the scale_pos_weight parameter to adjust the balance of positive and negative weights, can help the model better handle imbalanced datasets, which is the case here. By increasing the weight of positive examples, the model will learn to prioritize correctly classifying them, which should reduce the number of false negatives.

BD, I have done this before, but it would be better to use Average Precision(AP) instead of AUC, but it is better than other answers.

12-sep exam

Compensate for imbalance and overwriting.

B and E

B. Increase the XGBoost scale_pos_weight parameter to adjust the balance of positive and negative weights is the correct answer. According to https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost_hyperparameters.html, scale_pos_weight controls the balance of positive and negative weights. It's useful for unbalanced classes.