You are designing a data processing pipeline. The pipeline must be able to scale automatically as load increases. Messages must be processed at least once and must be ordered within windows of 1 hour. How should you design the solution?
A.
Use Apache Kafka for message ingestion and use Cloud Dataproc for streaming analysis.
B.
Use Apache Kafka for message ingestion and use Cloud Dataflow for streaming analysis.
C.
Use Cloud Pub/Sub for message ingestion and Cloud Dataproc for streaming analysis.
D.
Use Cloud Pub/Sub for message ingestion and Cloud Dataflow for streaming analysis.
Kafka is recommended over Pub/Sub only when there is requirement of high throughput, complex streaming, more flexibility with customization and fine-tuning configurations or when application spans multiple cloud providers and requires more flexibility in deployment across different platforms.
Data proc is serverbased
Dataflow is serverless which is used to run pipelines which uses apache framework in the background. Just
need to mention the number of workers needed.
so question saying we need scale automatically . so dataproc eliminate ho gaya
now Dataflow is correct , pub/sub is recommended for this scenario. D
Careful doing that: I got a question where you had to choose between Kafka and Pub/Sub... and the solution required to be able to replay all messages without time limit. So no Pub/Sub there.
This being a Google cert does not mean that they always force Google solutions.
A voting comment increases the vote count for the chosen answer by one.
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one.
So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
madhu1171
Highly Voted 4 years, 1 month ago[Removed]
Highly Voted 4 years, 1 month agogrshankar9
Most Recent 3 months, 1 week agoNeoNitin
8 months, 3 weeks agodconesoko
1 year, 4 months agozellck
1 year, 4 months agoPime13
1 year, 9 months agoVictorBa
2 years agomedeis_jar
2 years, 3 months agoMaxNRG
2 years, 3 months agohendrixlives
2 years, 4 months agoJG123
2 years, 5 months agoChelseajcole
2 years, 7 months agohendrixlives
2 years, 4 months agosandipk91
2 years, 8 months agoawssp12345
2 years, 9 months agosumanshu
2 years, 10 months agoSush12
3 years, 2 months ago