Exam DP-100 topic 3 question 63 discussion

Answers looks correct to me Difference between Erros: 105-095: 10 200-085: 115 250-100: 150 105-100: 5 ---> This is the best H value. Agree with #4 for Q1 400-050: 350 -> Highest Diff. So Poor for Q2

Why not 1 with lowest value in both training and validation?

used ChatGpt 4 and got this explanation which I agreed: The best hyperparameters to select would be the ones that have the lowest validation error, as this indicates how well the model is likely to perform on unseen data. In this case, that would be the one with a validation error of 50. The poorest training result would be the one with the highest training error. In this case, that would be the one with a training error of 400. In general, the goal of hyperparameter tuning is to minimize the validation error, which indicates how well the model is likely to perform on unseen data. The model with the lowest variance isn't necessarily the best model. A model with high bias can have low variance, but still be inaccurate. Similarly, a model with low bias can have high variance, but still be accurate. This is known as the bias-variance tradeoff.

Based on these values, the optimal hyperparameter setting seems to be H1. It has the lowest total error when you consider both the training and validation error, which suggests it may be the best compromise between underfitting and overfitting. The hyperparameter setting that displays the poorest training result would be H5, as it has the highest training error (TE=400), suggesting it might be underfitting to the training data.

chatGcP said : The best results are those that have a low training error and a low validation error. In the first case, the training error is 105 and the validation error is 95, while in the second case the training error is 105 and the validation error is 100. Therefore, the first case is better, as the validation error is lower. This indicates that the model is generalizing well and is not "memorizing" the training data.

The answer should be 1 and 5. When training/testing a model, the problem of overfitting and underfitting need to be considered. In the case of the best H value. H = 1 clearly produced the best model with minimum validation error on the test dataset (which is the dataset we care about).

Poorest training result --> 5 Best H Parameter, this question does not have enough information, we do not know the sample size for training and test data, If there are both in millions, then no one cares about 100 errors vs 500 errors, if they are only in thousands, then I will only consider 1 and 4, in this case I guess 4 is given slight better results in testing, so I will go 4

The question is on stack exchange https://stats.stackexchange.com/questions/570322/how-to-choose-a-models-hyperparameters-in-terms-of-the-variance/570485#570485

The answer is 1 and 5... Why would you choose an option with the two closest error? Would you choose 300 and 299 as the best ?

Agree with pepmir. 4 has least difference between validation and training result, and box 2 is about "poorest training result" which is by data given, 5

Underfitting – Validation and training error high Overfitting – Validation error is high, training error low Good fit – Validation error low, slightly higher than the training error Unknown fit - Validation error low, training error 'high'

We have to see which model generalizes well on test data..clearly in option 1 difference of train and test is 10..while in option 4 difference is only 5. So 4th one may generalize well .When we do train and test split , our target is to have as close train and test error along with minimum error

the answer is 1 and 5 for those who calculate the difference between losses, if we have 500 and 498 we would then chose this option?

the best model is not 4, it is 1 looking at the difference between training and validation errors is not helpful - by this logic a model with TE 105 VE 110 would also be better than model 1. This is clearly not the case.

Pepmir's explanation is correct.

Why not opt 2

This question is a weird one. The training error here is much higher than the validation error. Usually it's the other way around. Depending on the degree of overfitting, the VE can be a lot higher than the TE, but almost never smaller than the TE. Still the general rule for such a question is: 1) Go for the H with the smallest VE 2) The H with the highest VE is the worst. Unfortunately here H=3 and H=4 are equally bad, so doesn't make sense to choose only one of them w