Exam Professional Data Engineer topic 1 question 61 discussion

B. (Hadoop/Spark jobs are run on Dataproc, and the pre-emptible machines cost 80% less)

I think the answer should be B: https://cloud.google.com/dataproc/docs/concepts/compute/preemptible-vms

I believe it might be "B", but what if the job is mission critical? Pre-emptible VMs would be of no use.

I believe Exam Topics ought to provide brief explanation or supporting link to picked correct answers such as this one. Option A may be correct from the view point that Dataflow is a Serverless service that is fast, cost-effective and the fact that Preemptible VMs though can give large price discount may not always be available. It will be great to know the reason(s) behind Exam Topic selected option.

B. Use pre-emptible virtual machines (VMs) for the cluster Using pre-emptible VMs allows you to take advantage of lower-cost virtual machine instances that may be terminated by Google Cloud after a short period of time, typically after 24 hours. These instances can be a cost-effective way to handle workloads that can be interrupted, such as batch processing jobs like the one described in the question. Option A is not ideal, as it would require you to migrate the workload to Google Cloud Dataflow, which may cause additional complexity and would not address the issue of cost optimization. Option C is not ideal, as it would require you to use a higher-memory node which would increase the cost. Option D is not ideal, as it would require you to use SSDs on the worker nodes which would increase the cost. Using pre-emptible VMs is a better option as it allows you to take advantage of lower-cost virtual machine instances and handle workloads that can be interrupted, which can help to optimize the cost of the cluster.

What is happening with this test "correct answer" a lot of times it doesn't make any sense. As this one... Clear it's B

Using preemtible machines are cost effective , and because is suitable for a job mentioned here as it is fault tolerant .

User Pre-emptible VM machine and save process cost, and question want simple solution.

A- Data flow it's not cost-effective in comparison with dataproc B- Preemptible VM instances are available at much lower price—a 60-91% discount—compared to the price of standar, so this is the answer C and D are more expensive.

B is right way to go

Preemptible workers are the default secondary worker type. They are reclaimed and removed from the cluster if they are required by Google Cloud for other tasks. Although the potential removal of preemptible workers can affect job stability, you may decide to use preemptible instances to lower per-hour compute costs for non-critical data processing or to create very large clusters at a lower total cost https://cloud.google.com/dataproc/docs/concepts/compute/secondary-vms

B is teh right answer. examtopics update your answers or make your site free again.

B is right answer. my experience is not good with Examtopics, so many wrong answers.

B should be the right answer. I am amazed that almost 60% of the marked answers on the site are wrong.

Ans : B, here we are checking on reducing cost, so pre-emptiable machines are best choice

"this workload can run in approximately 30 minutes on a 15-node cluster," so you need performance for only 30 mins -> preemptible VMs https://cloud.google.com/dataproc/docs/concepts/compute/preemptible-vms

A is not valid, for apache spark jobs dataproc y the best choice. C and D are not correct, that might speed up the job or not. For sure if we use pre-emptible machines this will be cheaper and since we don’t have severe time restriction…thats the one. B