Exam AWS Certified Machine Learning - Specialty topic 1 question 40 discussion

THE ANSWER SHOULD BE B. YOU DON'T NEED TO THROUGH LAMBDA TO INTERGE CLOUDTRAIL Log Amazon SageMaker API Calls with AWS CloudTrail https://docs.aws.amazon.com/sagemaker/latest/dg/logging-using-cloudtrail.html

Agreed B for the following reasons # CloudTrail logs captured in S3 without any code/lambda # The custom metrics can be published to Cloudwatch...in this case it would be a test for overfit on MXNET .... which will set off an alarm .... which can then be subscribed on SNS

Breakdown of the Chosen Solution (B) Use AWS CloudTrail to log SageMaker API calls to Amazon S3 ✅ CloudTrail automatically logs all AWS API activity, including SageMaker API calls, for auditing. S3 stores these logs securely for auditor review. Push custom metrics to Amazon CloudWatch ✅ Model overfitting can be detected using a custom CloudWatch metric (e.g., validation loss increasing while training loss decreases). The SageMaker training script can push loss values to CloudWatch during training. Create a CloudWatch alarm + SNS notification ✅ Set a CloudWatch alarm on the overfitting metric (e.g., validation loss surpassing a threshold). Use Amazon SNS to send a notification (email, SMS, or Lambda trigger) when the alarm is triggered.

Option D involves using AWS CloudTrail to log Amazon SageMaker API calls to Amazon S3 and setting up Amazon SNS to receive a notification when the model is overfitting. While this approach addresses the logging requirement, it does not provide a mechanism for pushing custom metrics to Amazon CloudWatch, which is necessary for monitoring model performance and detecting overfitting. So 'B ' is correct

https://docs.aws.amazon.com/fr_fr/sagemaker/latest/dg/training-metrics.html#define-train-metrics It detects hardware resource usage issues (such as CPU, GPU, and I/O bottlenecks) and non-convergent model issues (such as overfitting, disappearing gradients, and tensor explosion). why couldn't the answer be D, as this covers all of the requirements, and B seems to add an extra step with adding push code, when it already has a builtin metric for overfitting.

A. NO - CloudTrail has built-in SageMaker API calls tracking, no lambda needed B. YES - the chain works C. NO - CloudTrail has built-in SageMaker API calls tracking, no lambda needed D. NO - CloudTrail has not specific Amazon SageMaker integration to detect overfitting

Option B

"least amount of code and fewest steps?" I think it's D.

I would consider D as well. You can just setup a SNS that is triggered by a built-in action like here: https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-built-in-actions.html You can see that overfitting is a built-in rule for MXNet from here: https://docs.aws.amazon.com/sagemaker/latest/dg/debugger-built-in-rules.html Not that B is not working. Maybe the question was prior to this new solution.

It's B.

AWS CloudTrail provides a history of AWS API calls made on the account. The Machine Learning team can use AWS CloudTrail to log Amazon SageMaker API calls to Amazon S3. They can then use CloudWatch to create alarms and receive notifications when the model is overfitting. To ensure auditors can view the Amazon SageMaker log activity report, the team can add code to push a custom metric to Amazon CloudWatch. This provides a single place to view and analyze logs across all the services and resources in the environment.

B. cloudwatch + metrics from sagemaker + sns https://docs.aws.amazon.com/fr_fr/sagemaker/latest/dg/training-metrics.html#define-train-metrics

B requires the least amount of code and satisfies all conditions

What does this line do? "Add code to push a custom metric to Amazon CloudWatch"

Its not B. Why would you use CloudTrail? Having used Lambda for API calls I'm inclined to agree with the original answer, C.

B it is

B it is