Exam Professional Machine Learning Engineer topic 1 question 40 discussion

I think should CE. I can't find any reference regarding B can reduce tuning time.

Answer: B & C (Ref: https://cloud.google.com/ai-platform/training/docs/using-hyperparameter-tuning) (A) Decreasing the number of parallel trials will increase tuning time. (D) Bayesian search works better and faster than random search since it's selective in points to evaluate and uses knowledge of previouls evaluated points. (E) maxTrials should be larger than 10*the number of hyperparameters used. And spanning the whole minimum space (10*num_hyperparams) already takes some time. So, lowering maxTrials has little effect on reducing tuning time.

In the PMLE book it's grid search instead of Bayesian search and that makes sense, but there is also marked Decrease the number of parallel trials as correct answer, which I think should be wrong.

With Vertex AI hyperparameter tuning, you can configure the number of trials and the search algorithm as well as range of parameters.

C) and D)

see pawan94

C and E, if you reference the latest docs of hptune job on vertex ai : 1. A not possible (refer: https://cloud.google.com/vertex-ai/docs/training/using-hyperparameter-tuning#:~:text=the%20benefit%20of%20reducing%20the%20time%20the) , if you reduce the number of parallel trials then the speed of overall completion gets negatively affected. . The question is about how to speed up the process but not changing the model params. Changing the optimization algorithm would lead to unexpected results. So in my opinion C and E ( after carefully reading the updated docs) and please don't believe everything CHATGPT says . I encountered so many questions where the LLM's are giving completely wrong answers

I chose C and D

Chat GPT says: . Set the early stopping parameter to TRUE. Early Stopping: Enabling early stopping allows the tuning process to terminate a trial if it becomes clear that it's not producing promising results. This prevents wasting time on unpromising trials and can significantly speed up the hyperparameter tuning process. It helps to focus resources on more promising parameter combinations. D. Change the search algorithm from Bayesian search to random search. Random Search Algorithm: Random search, as opposed to Bayesian optimization, doesn't attempt to build a model of the objective function. While Bayesian search can be more efficient in finding the optimal parameters, random search is often faster per iteration. Random search can be particularly effective when the hyperparameter space is large, as it doesn't require as much computational power to select the next set of parameters to evaluate.

C&E This video explains very well the max trials and parallel trials https://youtu.be/8hZ_cBwNOss This link explains early stopping See https://cloud.google.com/ai-platform/training/docs/using-hyperparameter-tuning#early-stopping

A increase time, B HP tuning job normally bottle neck is not at model size, D did reduce time, but might significantly hurt effectiveness

Running parallel trials has the benefit of reducing the time the training job takes (real time—the total processing time required is not typically changed). However, running in parallel can reduce the effectiveness of the tuning job overall. That is because hyperparameter tuning uses the results of previous trials to inform the values to assign to the hyperparameters of subsequent trials. When running in parallel, some trials start without having the benefit of the results of any trials still running. You can specify that AI Platform Training must automatically stop a trial that has become clearly unpromising. This saves you the cost of continuing a trial that is unlikely to be useful. To permit stopping a trial early, set the enableTrialEarlyStopping value in the HyperparameterSpec to TRUE.

Went with C & E

To speed up the tuning job without significantly compromising its effectiveness, you can take the following actions: A. Decrease the number of parallel trials: By reducing the number of parallel trials, you can limit the amount of computational resources being used at a given time, which may help speed up the tuning job. However, reducing the number of parallel trials too much could limit the exploration of the parameter space and result in suboptimal results. D. Change the search algorithm from Bayesian search to random search: Bayesian optimization is a computationally intensive method that requires more time and resources than random search. By switching to a simpler method like random search, you may be able to speed up the tuning job without compromising its effectiveness. However, random search may not be as efficient in finding the best hyperparameters as Bayesian optimization.

Early stopping is for training, not hyperparameter tuning

The two actions that can speed up hyperparameter tuning without compromising effectiveness are decreasing the number of parallel trials and changing the search algorithm from Bayesian search to random search.

B. Decrease the range of floating-point values: Reducing the range of the hyperparameters will decrease the search space and the time it takes to find the optimal hyperparameters. However, if the range is too narrow, it may not be possible to find the best hyperparameters. C. Set the early stopping parameter to TRUE: Setting the early stopping parameter to true will stop the trial when the performance has stopped improving. This will help to reduce the number of trials needed and thus speed up the hypertuning job without compromising its effectiveness. D.Changing the search algorithm from Bayesian search to random search could also be a valid action to speed up the hypertuning job. Random search can explore the hyperparameter space more efficiently and with less computation cost compared to Bayesian search, especially when the search space is large and complex. However, it may not be as effective as Bayesian search in finding the best hyperparameters in some cases.