Exam Professional Machine Learning Engineer topic 1 question 189 discussion

My answer is D.

https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

The choice is between B and D, both good BUT: Importing and making batch predictions is quite straightforward in BQ ML https://cloud.google.com/bigquery/docs/making-predictions-with-imported-tensorflow-models if not pre-processing needed on the data. If we need a more complete pipeline I'd chose D, but the tables need partitioning (100GB is the limit in Vertex AI): https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

My Answer: D The historical dataset is stored in BigQuery, which can be directly accessed by Vertex AI. Vertex AI offers batch prediction capabilities, allowing you to apply the model to the data stored in BigQuery without the need to export it. So, This approach leverages the scalability of Google Cloud infrastructure and avoids unnecessary data movement, being not necessary to export data to Cloud Store (options A and C), nor Import the TensorFlow model to BQ (option B).

https://cloud.google.com/bigquery/docs/making-predictions-with-imported-tensorflow-models#:~:text=Import%20TensorFlow%20models,-To%20import%20TensorFlow&text=In%20the%20Google%20Cloud%20console%2C%20go%20to%20the%20BigQuery%20page.&text=In%20the%20query%20editor%2C%20enter,MODEL%20statement%20like%20the%20following.&text=The%20preceding%20query%20imports%20a,BigQuery%20ML%20model%20named%20imported_tf_model%20.

https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tensorflow#limitations

Has to be B, because D has limitations: BigQuery data source tables must be no larger than 100 GB. https://cloud.google.com/vertex-ai/docs/tabular-data/classification-regression/get-batch-predictions#input_data_requirements

Same platform as data == less computation required to load and pass it to model

It could either be B or D. It seems like most of the limitations of B are mentioned in the problem (https://cloud.google.com/bigquery/docs/reference/standard-sql/bigqueryml-syntax-create-tensorflow#limitations) but some of them are not and we are left questioning if the model will match the remaining requirements. Therefore, I would go for D, which can import data from BigQuery. https://cloud.google.com/vertex-ai/docs/predictions/get-batch-predictions#bigquery

Limitations of other options: A and C. Exporting data: Exporting 10 TB of data to Cloud Storage incurs additional storage costs, transfer time, and potential data management complexities. B. BigQuery ML: While BigQuery ML supports some TensorFlow models, it might have limitations with certain model architectures or features. Additionally, it might not be as optimized for large-scale batch inference as Vertex AI.