Exam Professional Cloud Architect topic 8 question 4 discussion

It's Pub/Sub, too much data streaming for Bigquery...

Its B, it exceeds the streaming limit for BQ

Has to be pub-sub, you have remote vehicles and need to guarantee message delivery.

Wow its almost like GCP shouldnt have offloaded their IoT Core product - you cant "Write direct to PubSub". Its the correct answer but its overly simplified Writing directly to GCS will cost a fortune to retrieve in GET requests etc

Streamed data is available for real-time analysis within a few seconds of the first streaming insertion into a table. Instead of using a job to load data into BigQuery, you can choose to stream your data into BigQuery one record at a time by using the tabledata().insertAll() method. This approach enables querying data without the delay of running a load job. References: https://cloud.google.com/bigquery/streaming-data-into-bigquery

They are sending files through FTP why everyone is missing this point? The max message size in pub sub is 10MB as I remember, I would keep the files solution and try to roll out updates to direct the upload to GCS

So many people pointing out this breaks the BigQuery quota limit but very few pointing out it also breaks the Pub/Sub quote limit.......... So the answer is either not bound by the quota limit (in which case why not BigQuery) both are wrong and we stick with FTP

it's B

I know it's B, however the sensors are probably legacy systems, that can not communicate to a pub/sub queue. Ignoring how huge is to change or adapta 20 million devices is a mistake.

To handle the volume of data that TerramEarth plans to ingest, it is recommended to use a scalable and reliable data ingestion solution such as Google Cloud Pub/Sub. With Cloud Pub/Sub, the vehicles can stream data directly to the service, which can handle the high volume of data and provide a buffer to absorb sudden spikes in traffic. The data can then be processed and stored in a data warehouse such as BigQuery for analysis. Option A (writing data directly to GCS) may not be suitable for handling high volumes of data in real-time and may result in data loss if the volume exceeds the capacity of GCS. Option C (streaming data directly to BigQuery) may not be suitable for handling high volumes of data in real-time as it may result in data loss or ingestion delays. Option D (continuing to write data using the existing system) may not be suitable as the current system may not be able to handle the increased volume of data and may result in data loss or ingestion delays.

B is the correct answer

ok for B

B is the correct answer, this similar question was in google simple questions

B is right!

Should be A - see next question on 80% cellular connectivity and Avro format files streamed directly to GCS

We need to buffer, the default limit of BigQuery is 100 API calls per second, till now this cannot be changed. Hence we should ease using Pub/Sub so B.

You can request limit increases to use BQ streaming for this load, but why pay to store data before ETL?