Exam AWS Certified Machine Learning Engineer - Associate MLA-C01 topic 1 question 18 discussion

A. No, early stopping is for preventing overfitting B. No, increasing test will not help with oscillating loss C. No, increasing learning rate will make things worsening D. Oscillating loss in training is a sign that the training is not converging, this can happen when learning rate is too high. Reducing learning rate will help here

Oscillating patterns in a train/validation loss shows it's not converging to a minima. Low learning rate will make it converge.

The oscillating loss during training is a clear sign that the learning rate is too high. Reducing the learning rate will stabilize the optimization process, allowing the model to converge smoothly.

The oscillating pattern of training and validation loss indicates that the learning rate is too high. A high learning rate causes the model to overshoot the optimal point in the loss landscape, leading to oscillations instead of convergence. Reducing the learning rate allows the model to make smaller, more precise updates to the weights, improving convergence. A. Early stopping prevents overfitting by halting training when validation performance stops improving. However, it does not address the root cause of oscillating loss. B. The size of the test set does not affect the training dynamics or loss patterns. C. Increasing the learning rate would worsen the oscillations and prevent the model from converging.

oscillating = decrease the learning rate