ExamTopics Logo
Mail Us[email protected]
Menu
Google Discussions
exam questions

Exam Professional Data Engineer All Questions

View all questions & answers for the Professional Data Engineer exam
Go to Exam

Exam Professional Data Engineer topic 1 question 278 discussion

Actual exam question from Google's Professional Data Engineer
Question #: 278
Topic #: 1
[All Professional Data Engineer Questions]

Your car factory is pushing machine measurements as messages into a Pub/Sub topic in your Google Cloud project. A Dataflow streaming job, that you wrote with the Apache Beam SDK, reads these messages, sends acknowledgment to Pub/Sub, applies some custom business logic in a DoFn instance, and writes the result to BigQuery. You want to ensure that if your business logic fails on a message, the message will be sent to a Pub/Sub topic that you want to monitor for alerting purposes. What should you do?

  • A. Enable retaining of acknowledged messages in your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the subscription/num_retained_acked_messages metric on this subscription.
  • B. Use an exception handling block in your Dataflow’s DoFn code to push the messages that failed to be transformed through a side output and to a new Pub/Sub topic. Use Cloud Monitoring to monitor the topic/num_unacked_messages_by_region metric on this new topic.
  • C. Enable dead lettering in your Pub/Sub pull subscription, and specify a new Pub/Sub topic as the dead letter topic. Use Cloud Monitoring to monitor the subscription/dead_letter_message_count metric on your pull subscription.
  • D. Create a snapshot of your Pub/Sub pull subscription. Use Cloud Monitoring to monitor the snapshot/num_messages metric on this snapshot.
Show Suggested Answer Hide Answer
Suggested Answer: B 🗳️
by scaenruy at Jan. 4, 2024, 5:24 a.m.

Comments

Chosen Answer:
This is a voting comment (?). It is better to Upvote an existing comment if you don't have anything to add.
Switch to a voting comment New
Submit Cancel
raaad
Highly Voted 1 year, 3 months ago
Selected Answer: B
- Exception Handling in DoFn: Implementing an exception handling block within DoFn in Dataflow to catch failures during processing is a direct way to manage errors. - Side Output to New Topic: Using a side output to redirect failed messages to a new Pub/Sub topic is an effective way to isolate and manage these messages. - Monitoring: Monitoring the num_unacked_messages_by_region on the new topic can alert you to the presence of failed messages.
upvoted 15 times
...
chrissamharris
Most Recent 7 months ago
Option C - dead letter topic is built in and requires no changes https://cloud.google.com/pubsub/docs/handling-failures Enable dead lettering in your Pub/Sub pull subscription, and specify a new Pub/Sub topic as the dead letter topic. Use Cloud Monitoring to monitor the subscription/dead_letter_message_count metric on your pull subscription.
upvoted 2 times
Positron75
5 months ago
Dead lettering is used to handle messages that have not been acknowledged, but that's unrelated to the processing that Dataflow does, which takes place later in the chain. A message could still be acknowledged and fail processing for whatever reason, so it would not be sent to the dead letter topic. Also, Google advises against using dead lettering with Dataflow anyway: https://cloud.google.com/dataflow/docs/concepts/streaming-with-cloud-pubsub#dead-letter-topics Correct answer is B. The error handling has to be written into the Dataflow pipeline itself.
upvoted 2 times
...
...
7787de3
7 months, 2 weeks ago
Selected Answer: B
See here: https://cloud.google.com/dataflow/docs/concepts/streaming-with-cloud-pubsub#unsupported-features It's not recommended to use Pub/Sub dead-letter topics with Dataflow (...) Instead, implement the dead-letter pattern explicitly in the pipeline
upvoted 1 times
...
Jeyaraj
9 months, 1 week ago
Option B. Here's why: Side Output for Failed Messages: Dataflow allows you to use side outputs to handle messages that fail processing. In your DoFn , you can catch exceptions and write the failed messages to a separate PCollection . This PCollection can then be written to a new Pub/Sub topic. New Pub/Sub Topic for Monitoring: Creating a dedicated Pub/Sub topic for failed messages allows you to monitor it specifically for alerting purposes. This provides a clear view of any issues with your business logic. topic/num_unacked_messages_by_region Metric: This Cloud Monitoring metric tracks the number of unacknowledged messages in a Pub/Sub topic. By monitoring this metric on your new topic, you can identify when messages are failing to be processed correctly.
upvoted 1 times
...
joao_01
1 year ago
I would like to know why isn't anyone considering the option C.
upvoted 1 times
joao_01
1 year ago
I think that C is not right anyways: In order to use dead_letter feature, the message CANNOT be acknowledge (somehow) by the subscriber. In this question it says that the messages are first acknowledge and then applied the business logic. So, if there are error in the business logic we cannot use the feature dead_letter, beacuse the message was already acknowledge. Thus, option B is the right one.
upvoted 2 times
...
...
hanoverquay
1 year, 1 month ago
Selected Answer: B
option B
upvoted 1 times
...
JyoGCP
1 year, 2 months ago
Selected Answer: B
Option B
upvoted 1 times
...
Matt_108
1 year, 3 months ago
Selected Answer: B
Option B - Raaad explanation is complete
upvoted 1 times
...
scaenruy
1 year, 3 months ago
Selected Answer: B
B. Use an exception handling block in your Dataflow’s DoFn code to push the messages that failed to be transformed through a side output and to a new Pub/Sub topic. Use Cloud Monitoring to monitor the topic/num_unacked_messages_by_region metric on this new topic.
upvoted 1 times
...
Community vote distribution
A (35%)
C (25%)
B (20%)
Other
Most Voted
A voting comment increases the vote count for the chosen answer by one.

Upvoting a comment with a selected answer will also increase the vote count towards that answer by one. So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.

SaveCancel
exam
Someone Bought Contributor Access for:
SY0-701
London, 1 minute ago