Your company uses Cloud Logging to manage large volumes of log data. You need to build a real-time log analysis architecture that pushes logs to a third-party application for processing. What should you do?
A.
Create a Cloud Logging log export to Pub/Sub.
B.
Create a Cloud Logging log export to BigQuery.
C.
Create a Cloud Logging log export to Cloud Storage.
D.
Create a Cloud Function to read Cloud Logging log entries and send them to the third-party application.
A: Creating a Cloud Logging log export to Pub/Sub is the correct solution for this scenario. Pub/Sub is designed for real-time messaging and can push messages (in this case, log entries) to a third-party application for processing.
B: While BigQuery is great for analyzing large volumes of data, it's not designed for real-time data pushing to third-party applications.
D: Creating a Cloud Function to read log entries and send them to a third-party application could work, but it would add unnecessary complexity. Using Pub/Sub is a simpler and more efficient solution.
My answer is A.
Third party service is the one responsible for analytics.
From Google cloud we just need to push the log messages to a third party application for analytics that is the part of analytics architecture.
Real time push means, I go with Pub-sub.
https://cloud.google.com/logging/docs/export/configure_export_v2#overview
https://cloud.google.com/logging/docs/export/pubsub:
This document explains how you can find log entries that you routed from Cloud Logging to Pub/Sub topics, which occurs in near real-time. We recommend using Pub/Sub for integrating Cloud Logging logs with third-party software.
When you route logs to a Pub/Sub topic, Logging publishes each log entry as a Pub/Sub message as soon as Logging receives that log entry. Routed logs are generally available within seconds of their arrival to Logging, with 99% of logs available in less than 60 seconds.
The processing will be done in a third-party application so we need a solution to pass logs from gcp to thirs party in real time and no need for analytics. So the solution is pub/sub.
Example on a case corresponding to the question by google:
https://cloud.google.com/architecture/exporting-stackdriver-logging-for-splunk
no need for analytics??? the question says "You need to build a real-time log analysis architecture that pushes logs to a third-party application for processing " its bigquery, it can connect to others cloud providers..https://cloud.google.com/bigquery/docs/introduction#bigquery-analytics
https://cloud.google.com/blog/products/data-analytics/bigquery-performance-powers-real-time-analytics
While BigQuery can be used for log analysis, it is not well suited for real-time log processing. BigQuery is designed for batch processing of large amounts of data and may not be able to provide the low latency and real-time processing capabilities required for real-time log analysis. Additionally, BigQuery may be more expensive than other options for real-time log analysis, as it charges for both storage and processing.
Therefore, for real-time log analysis, it is more appropriate to use a solution like Cloud Pub/Sub, which is specifically designed for real-time streaming of data.
My understanding is that third-party application may not be a GCP solution.
I would go for A
Answer B
Third party transfers for BigQuery Data Transfer Service allow you to automatically schedule and manage recurring load jobs for external data sources such as Salesforce CRM, Adobe Analytics, and Facebook Ads.
https://cloud.google.com/bigquery/docs/introduction#bigquery-analytics
https://cloud.google.com/blog/products/data-analytics/bigquery-performance-powers-real-time-analytics
Pub/sub does real time streaming not analytics. analytics its biquery and dataflow those can do realtime analytics.
A is the only option that meets all of these requirements:
- Handles large volumes of log data
- Sends messages (logs) to 3rd party applications in real time
Can pub/sub analyse data?? Kindly revisit the documentation, the question says You need to build a real-time log analysis architecture, not real time streaming, pub/sub does realtime streaming not analysis, so its bigquery , i dnt know if you practically worked on gcp then you will know and understand these solutions
Third party transfers for BigQuery Data Transfer Service allow you to automatically schedule and manage recurring load jobs for external data sources such as Salesforce CRM, Adobe Analytics, and Facebook Ads.
to do thrid party transfers bigquery has this above mentioned capability
Third party transfers for BigQuery Data Transfer Service allow you to automatically schedule and manage recurring load jobs for external data sources such as Salesforce CRM, Adobe Analytics, and Facebook Ads.
you cant analyse data on pub/sub but you stream, so understand the difference, answer is Bigquery
B is the answer. " real time log analysis csan be done by Bigquery, C isnt correct.
upvoted 2 times
...
...
Log in to ExamTopics
Sign in:
Community vote distribution
A (35%)
C (25%)
B (20%)
Other
Most Voted
A voting comment increases the vote count for the chosen answer by one.
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one.
So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
alpha_canary
2 months, 2 weeks agoalpha_canary
2 months, 2 weeks ago__rajan__
9 months, 1 week agopurushi
10 months, 4 weeks agoPime13
1 year, 4 months agotelp
1 year, 5 months agoTNT87
1 year, 5 months agomrvergara
1 year, 4 months agoTNT87
1 year, 6 months agox_cath
1 year, 6 months agoTNT87
1 year, 6 months agoTNT87
1 year, 6 months agosharath25
1 year, 6 months agojcataluna
1 year, 6 months agox_cath
1 year, 6 months agoTNT87
1 year, 6 months agozellck
1 year, 6 months agotest010101
1 year, 6 months agoTNT87
1 year, 6 months agomelisargh
1 year, 7 months agoTNT87
1 year, 6 months ago