Exam Professional Machine Learning Engineer topic 1 question 164 discussion

sounds to me that the new model has too few requests per hour and therefore scales downs to 0. Which means it has to create the an instance every time it serves a request, and this takes time. By manually setting the number of nodes, the nodes will always be running, whether or not they are serving predictions

The new model version is failing badly, with response times up to 180 seconds being unacceptable for an online service. The absolute first priority is to stop impacting users. Removing the new model from the canary test and routing all traffic back to the stable, existing version immediately mitigates the problem. Once the production environment is stable, you can begin your root cause analysis. The problem is a performance bottleneck, not an issue with scaling or infrastructure. Since the only thing that changed was the retrained model, the cause must lie within the new model artifact or its prediction code. Comparing the new version with the old is the most logical way to find what changed to cause the drastic slowdown.

Need to check the performance bottlenecks

You're performing 20 predictions an hour - so scaling isn’t the root issue. Code issue.

Since the same model is being used and the only change is the data, it's likely that the latency issue is caused by how Vertex AI is scaling the prediction service.

Removing the new model from production to debug and address the root cause of the latency issue is the most efficient and logical course of action. This ensures minimal disruption to production services and lays the groundwork for a smooth rollout after fixing the bottleneck

I don't see B as the right answer. The Vertex AI Endpoint cannot scale to 0 for newer version of the model. > When you configure a DeployedModel, you must set dedicatedResources.minReplicaCount to at least 1. In other words, you cannot configure the DeployedModel to scale to 0 prediction nodes when it is unused. https://cloud.google.com/vertex-ai/docs/general/deployment#scaling

B can be effective in controlling resources available to the new model, ensuring that it is not delayed by the autoscaling trying to scale up from 0. Not A: there is no indication in the description that quota limits cause the slowdown and does not address issue where new model is performing poorly on canary testing. Not C : when you pull the new model from prod environment, you could affect end-user experience Not D: Same as C plus you rely on batch predictions which does not align with the need for online, real-time predictions in the prod environemnt. Data Labeling Service is more about assessing accuracy and less about resolving latency issues.

You have retrained the same model on a new batch of data

bottleneck seems to be start of node as there are very low number of requests so having one node always available will help in this case.

went with c

I went C. Diagnosing the root cause.

Choose C. The significant increase in response time from 1 second to between 30 and 180 seconds indicates a performance issue with the new model. Before making any further changes or decisions, it's crucial to identify the root cause of this performance bottleneck. By comparing the code of the new model with the existing model, you can pinpoint any differences that might be causing the slowdown. In A, This may not be the root cause and could incur unnecessary costs without addressing the performance issue. In B,  it doesn't address the underlying issue causing the significant increase in response time observed during canary testing. in D, This would significantly increase latency and hinder real-time predictions, negatively impacting user experience.

Unsure on this one, but I would go with A. B. Turning off auto-scaling is a good measure when dealing with datasets with steep spikes of requests traffic (here we are dealing with avg. 20 request per hour) "The service may not be able to bring nodes online fast enough to keep up with large spikes of request traffic." https://cloud.google.com/blog/products/ai-machine-learning/scaling-machine-learning-predictions C. You retrain the SAME model on a different batch of data. It is implied that the code is the same too? D. Actual quality of the model is not in question here, but rather the long prediction time per request. Even if the requests traffic is very low, I can only consider option A: the selected quota cannot deal with the amount of concurrent prediction requests.

Option B or D is completely wrong. Option A to raise the quota might be necessary in some situations but doesn't necessarily deal with the performance issue at the test. Option C seems like the most suitable option.