ExamTopics Logo
Mail Us[email protected]
Menu
Amazon Discussions
exam questions

Exam AWS Certified Machine Learning - Specialty All Questions

View all questions & answers for the AWS Certified Machine Learning - Specialty exam
Go to Exam

Exam AWS Certified Machine Learning - Specialty topic 1 question 150 discussion

Exam question from Amazon's AWS Certified Machine Learning - Specialty
Question #: 150
Topic #: 1
[All AWS Certified Machine Learning - Specialty Questions]

A bank wants to launch a low-rate credit promotion. The bank is located in a town that recently experienced economic hardship. Only some of the bank's customers were affected by the crisis, so the bank's credit team must identify which customers to target with the promotion. However, the credit team wants to make sure that loyal customers' full credit history is considered when the decision is made.
The bank's data science team developed a model that classifies account transactions and understands credit eligibility. The data science team used the XGBoost algorithm to train the model. The team used 7 years of bank transaction historical data for training and hyperparameter tuning over the course of several days.
The accuracy of the model is sufficient, but the credit team is struggling to explain accurately why the model denies credit to some customers. The credit team has almost no skill in data science.
What should the data science team do to address this issue in the MOST operationally efficient manner?

  • A. Use Amazon SageMaker Studio to rebuild the model. Create a notebook that uses the XGBoost training container to perform model training. Deploy the model at an endpoint. Enable Amazon SageMaker Model Monitor to store inferences. Use the inferences to create Shapley values that help explain model behavior. Create a chart that shows features and SHapley Additive exPlanations (SHAP) values to explain to the credit team how the features affect the model outcomes.
  • B. Use Amazon SageMaker Studio to rebuild the model. Create a notebook that uses the XGBoost training container to perform model training. Activate Amazon SageMaker Debugger, and configure it to calculate and collect Shapley values. Create a chart that shows features and SHapley Additive exPlanations (SHAP) values to explain to the credit team how the features affect the model outcomes.
  • C. Create an Amazon SageMaker notebook instance. Use the notebook instance and the XGBoost library to locally retrain the model. Use the plot_importance() method in the Python XGBoost interface to create a feature importance chart. Use that chart to explain to the credit team how the features affect the model outcomes.
  • D. Use Amazon SageMaker Studio to rebuild the model. Create a notebook that uses the XGBoost training container to perform model training. Deploy the model at an endpoint. Use Amazon SageMaker Processing to post-analyze the model and create a feature importance explainability chart automatically for the credit team.
Show Suggested Answer Hide Answer
Suggested Answer: B 🗳️
by spaceexplorer at April 30, 2022, 10:12 p.m.
Disclaimers:
  • - ExamTopics website is not related to, affiliated with, endorsed or authorized by Amazon.
  • - Trademarks, certification & product names are used for reference only and belong to Amazon.

Comments

Chosen Answer:
This is a voting comment (?). It is better to Upvote an existing comment if you don't have anything to add.
Switch to a voting comment New
Submit Cancel
spaceexplorer
Highly Voted 2 years, 12 months ago
Selected Answer: B
B, SageMaker Model Debugger is used to generate SHAP values
upvoted 15 times
siju13
2 years, 11 months ago
https://aws.amazon.com/blogs/machine-learning/ml-explainability-with-amazon-sagemaker-debugger/
upvoted 7 times
...
...
V_B_
Highly Voted 2 years, 8 months ago
Selected Answer: C
I believe C is the right answer, it is simpler and more accurate than B.
upvoted 6 times
DimLam
1 year, 6 months ago
It will show only importance of features not their contribution to the final score
upvoted 2 times
...
...
2eb8df0
Most Recent 1 month, 3 weeks ago
Selected Answer: A
The problem is at inference time, not training time. So its A
upvoted 1 times
...
amlgeek
6 months, 3 weeks ago
I hesitate between A and B... In the question, the credit team wants to understand the reason why the model denies credit at inference time, not at training time... Sagemaker Model Monitor compute SHAP values at inference time while Sagemaker Debugger compute SHAP values at training time... I'm leading more for A as an answer.
upvoted 2 times
...
kyuhuck
1 year, 2 months ago
Selected Answer: A
The best option is to use Amazon SageMaker Studio to rebuild the model and deploy it at an endpoint. Then, use Amazon SageMaker Model Monitor to store inferences and use the inferences to create Shapley values that help explain model behavior. Shapley values are a way of attributing the contribution of each feature to the model output. They can help the credit team understand why the model makes certain decisions and how the features affect the model outcomes. A chart that shows features and SHapley Additive exPlanations (SHAP) values can be created using the SHAP library in Python. This option is the most operationally efficient because it leverages the existing XGBoost training container and the built-in capabilities of Amazon SageMaker Model Monitor and SHAP library.
upvoted 4 times
...
loict
1 year, 7 months ago
Selected Answer: B
A. NO - too complicated to compute SHAP B. YES - Debugger supports built-in SHAP C. NO - too complicated to compute SHAP D. NO - too complicated to compute SHAP
upvoted 2 times
...
ccpmad
1 year, 9 months ago
Selected Answer: B
Option B utilizes Amazon SageMaker Studio to build and train the model, and it also activates Amazon SageMaker Debugger, which allows calculating and collecting Shapley values. These Shapley values will help explain accurately why the model denies credit to certain customers. Generating a chart that displays the features and their SHAP values will provide a visual and clear explanation of the impact of each feature on the model's decisions, making it easier for the credit team with limited data science skills to understand.
upvoted 1 times
...
Mickey321
1 year, 9 months ago
Either A or B Sage Maker Monitor require no experience so A is preferred while B can provide more details but depend if require knowledge to use it.
upvoted 2 times
Mickey321
1 year, 9 months ago
More towards B
upvoted 1 times
...
...
Ahmedhadi_
2 years ago
Selected Answer: A
SageMaker Model Monitor is a tool that helps monitor the quality of model predictions over time by analyzing data inputs and outputs during inference. It can detect and alert when data drift or concept drift occurs, and can identify features that are most responsible for the changes in model behavior. Model Monitor can be used to continuously monitor and improve model performance, and can be integrated with SageMaker endpoints or SageMaker Pipelines. SageMaker Debugger is a tool that helps debug machine learning models during training by analyzing the internal states of the model, such as weights and gradients, as well as the data inputs and outputs during training. It can detect and alert when common training issues occur, such as overfitting or underfitting, and can identify the root causes of these issues. Debugger can be used to improve model accuracy and convergence, and can be integrated with SageMaker training jobs.
upvoted 2 times
Ahmedhadi_
2 years ago
After reconsideration, it is actually B. https://aws.amazon.com/blogs/machine-learning/ml-explainability-with-amazon-sagemaker-debugger/
upvoted 2 times
...
...
Mllb
2 years ago
Selected Answer: B
Debugger because we are in the context of "training data"
upvoted 1 times
ZSun
1 year, 11 months ago
There are so many explanations, but most of them are just superfacial, focusing on what service is related to SHAP. This is the only one really answer the difference between A and C. 1. Both SagaMaker Model Monitor and Debugger can explain model, can generate SHAP. so it should be either A or C 2. Monitor is about inference. After deploy the model, we may find some attributes start to contribute more to the model, contradict to the training dataset. This case we use SageMaker Model Monitor. But our problem is not about deploying, is still in training stage. We only want to figure out why some customer with specfic characteristics are more likely to get loan, in other words, certain feature contribute more to the prediction. It is C !!!! If you don't fully understant the question, stop explaining !!!
upvoted 2 times
ZSun
1 year, 11 months ago
not comparing between A and C, should be A and B
upvoted 2 times
...
...
...
Amit11011996
2 years, 1 month ago
Selected Answer: C
C is the straight forward and simpler.
upvoted 1 times
...
Amit11011996
2 years, 1 month ago
Why not C? 'C' is the most easiest way to find out.!
upvoted 1 times
...
Chelseajcole
2 years, 1 month ago
Selected Answer: B
This is debugger’s work
upvoted 2 times
...
AjoseO
2 years, 2 months ago
Selected Answer: A
Option A suggests using Amazon SageMaker Model Monitor to store inferences and create Shapley values that can help explain the model's behavior. This option can be more operationally efficient because it doesn't require the credit team to understand the complexities of Shapley values, and it doesn't necessarily slow down the model's inference time.
upvoted 1 times
AjoseO
2 years, 2 months ago
After a review, I go with option B
upvoted 1 times
...
...
Community vote distribution
A (35%)
C (25%)
B (20%)
Other
Most Voted
A voting comment increases the vote count for the chosen answer by one.

Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.

SaveCancel
exam
Someone Bought Contributor Access for:
SY0-701
London, 1 minute ago