Exam Professional Machine Learning Engineer topic 1 question 176 discussion

Since it is already given that we will be using a TF-Model and do experiments exclusevly there, I don't see why we wouldn't use TF-Layers to preprocess the data. We would minimize costs by not having to store additional data. Time would be around the same as the layer transforms the attribute during training time and development would also be simpler, since if you are using keras it would literally be 2 more lines of code. However I see the Argument for B as well but I would still go with C in this case. Specifically in this case I would use Normalization layer for normalization and Discretization layer for binning/bucketing.

In-place Transformation: BigQuery allows you to perform data transformations directly within the data warehouse using SQL queries. This eliminates the need for data movement and reduces processing time compared to other options that involve data transfer. Minimized Development Effort: Since you're already familiar with SQL, writing queries for mm-max scaling and bucketing requires minimal additional development effort compared to learning and implementing new frameworks like Spark or Dataflow. Cost-Effective: BigQuery's serverless architecture scales processing power based on your workload. This can be more cost-effective than managing separate processing clusters like Dataproc.

B - Keeps the preprocessing algorithm seperate from the model

C This option allows you to leverage the power and simplicity of TensorFlow to preprocess and transform the data with simple Python code

BigQuery can do both transformations https://cloud.google.com/bigquery/docs/manual-preprocessing#numerical_functions

BigQuery (SQL) is the easiest, cheapest approach