Exam Professional Data Engineer topic 1 question 45 discussion

Answer: C

Answer: C Description: Scheduler for adhoc jobs – 3 jobs free and $0.10 per job

App Engine Cron is limited to scheduling tasks within your App Engine application, whereas Cloud Scheduler can trigger actions on various Google Cloud services like Cloud Functions, Pub/Sub topics, or external HTTP endpoints.

Service was renamed, but the answer is still - C

C. Using the Google App Engine Cron Service to run the Cloud Dataflow job allows you to automate the execution of the job. By creating a cron job, you can ensure that the Dataflow job is triggered exactly once per day at a specified time. This approach is automated, reliable, and fits the requirement of processing the log file once per day.

C. For a modern solution, Cloud Scheduler

Currently, Cloud Scheduler takes over the scheduling functions.

I don't understand why that dataflow is used for processing? even though it should be processed once per a day?? is it more suitable for processing by using Dataproc instead?

C was correct but nowadays you'd schedule a Dataflow job with Cloud Scheduler: https://cloud.google.com/community/tutorials/schedule-dataflow-jobs-with-cloud-scheduler

Correct Answer: C. ❌ A: Dataproc is a managed Apache Spark and Apache Hadoop service, makes no sense to use it ❌ B: This might sound as the cheapest, but is highly error prone, besides, anyone in charge of this has a salary and I doubt is a low one. ✅ C: This is the easiest/fastest/cheapest way to trigger job runs, you can even set retry attempts. source: https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml. ❌ D: Setting this would be much more expensive than the cron-job

Answer: C

Ans: C

I know probably this question is testing on if you know corn.yaml and its function in App Engine. But why B will be more expensive? Human capital cost? Let's say if hiring a person click the button will be cheaper than launch an app engine, should we reconsider B?

Scheduling Jobs with cron.yaml Free applications can have up to 20 scheduled tasks. Paid applications can have up to 250 scheduled tasks.

Vote for 'C'

Correct C

Answer is C. https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml