Exam Professional Data Engineer topic 1 question 171 discussion

A. When creating your model, use BigQuery's TRANSFORM clause to define preprocessing steps. At prediction time, use BigQuery's ML.EVALUATE clause without specifying any transformations on the raw input data. Using the TRANSFORM clause, you can specify all preprocessing during model creation. The preprocessing is automatically applied during the prediction and evaluation phases of machine learning. Reference: https://cloud.google.com/bigquery-ml/docs/bigqueryml-transform

A is the answer. https://cloud.google.com/bigquery-ml/docs/bigqueryml-transform Using the TRANSFORM clause, you can specify all preprocessing during model creation. The preprocessing is automatically applied during the prediction and evaluation phases of machine learning

C. Use a BigQuery view to define your preprocessing logic. When creating your model, use the view as your model training data. At prediction time, use BigQuery's ML.EVALUATE clause without specifying any transformations on the raw input data. Explanation: Preventing Data Skew: Training-serving skew occurs when the transformations applied to training data are not identically applied to prediction data. Using a BigQuery view ensures consistent preprocessing for both training and prediction. Advantages of BigQuery Views: Views encapsulate preprocessing logic, ensuring that the same transformations are applied whenever the view is queried. By referencing the view during both training and prediction, you eliminate the need for manual transformations and the risk of discrepancies.

A

The key to preventing skew in machine learning models is to ensure that the same data preprocessing steps are applied consistently to both the training data and the prediction data. In option B, the TRANSFORM clause in BigQuery ML is used to define preprocessing steps during model creation, and a saved query is used to apply the same transformations to the raw input data before making predictions. This ensures consistency and prevents skew. The ML.EVALUATE function is then used to evaluate the model’s performance on the transformed prediction data. This is the recommended workflow

Option A

A is correct answer if we use TRANSFORM clause in BigQuery no need to use any transform while evaluating and predicting https://cloud.google.com/bigquery/docs/bigqueryml-transform

A is the correct answer

Problem: Skew One thing that I overlooked when answering previously is that B, C does not address skew. When we preprocess our training data, we need to save our scaled factors somewhere, and when performing predictions on our test data, we need to use the scaling factors of our training data to predict the results. ML.EVALUATE already incorporates preprocessing steps for our test data using the saved scaled factors.

Stew prediction time by remove the preprocessing!

https://cloud.google.com/bigquery-ml/docs/bigqueryml-transform Ans A

This query's nested SELECT statement and FROM clause are the same as those in the CREATE MODEL query. Because the TRANSFORM clause is used in training, you don't need to specify the specific columns and transformations. They are automatically restored. Reference: https://cloud.google.com/bigquery-ml/docs/bigqueryml-transform