Exam Professional Data Engineer topic 1 question 130 discussion

Should be D. https://cloud.google.com/blog/products/gcp/performing-large-scale-mutations-in-bigquery

Should be D https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#merge_statement https://cloud.google.com/blog/products/gcp/performing-large-scale-mutations-in-bigquery

Should be D.

import all the data into a separate table and use that for updates is better than creating smaller csv which leads to more operational time to get it done and harder to manage it.

This limit was removed a long time ago already. Anyway, bulk imports are better.

D BigQuery DML statements have no quota limits. https://cloud.google.com/bigquery/quotas#data-manipulation-language-statements However, DML statements are counted toward the maximum number of table operations per day and partition modifications per day. DML statements will not fail due to these limits. In addition, DML statements are subject to the maximum rate of table metadata update operations. If you exceed this limit, retry the operation using exponential backoff between retries.

there is no update quota anymore. but i would say D

Option D is the right answer

No DML limits from 3rd march 2020 But if the questions is given in the exam, choose D asfor the options A, B,C as they are speaking about the limitations of the DML Limits. Atleast, D is giving an alternative!

Is this question still valid? What about DML without limits? https://cloud.google.com/blog/products/data-analytics/dml-without-limits-now-in-bigquery

D: BigQuery is primarly designed and suit to append-only technology with some limited DML statements. It's not a relational database where you constantly update your user records if they edit their profile. Instead you need to arhitect your code so each edit is a new row in Bigquery, and you always query the latest row. The DML statement limitation is low, because it targets different scenarios and not yours, aka live update on rows. You could ingest your data into a separate table, and issue 1 update statement per day. https://stackoverflow.com/questions/45183082/can-we-increase-update-quota-in-bigquery https://cloud.google.com/blog/products/gcp/performing-large-scale-mutations-in-bigquery

https://cloud.google.com/bigquery/docs/reference/standard-sql/dml-syntax#merge_statement https://cloud.google.com/blog/products/gcp/performing-large-scale-mutations-in-bigquery

old question I guess, should not be in the exam anymore (?) https://cloud.google.com/blog/products/data-analytics/dml-without-limits-now-in-bigquery

Vote for D

D: https://cloud.google.com/blog/products/gcp/performing-large-scale-mutations-in-bigquery

D: https://cloud.google.com/blog/products/bigquery/performing-large-scale-mutations-in-bigquery

D should be the answer. Avoid updates in Datawarehousing environment instead use merge to create a new table.