Exam Professional Machine Learning Engineer topic 1 question 83 discussion

B) Pub/Sub & DataFlow

The simplest solution that can support an eventual batch prediction (triggered by pub/sub) even the semi-real time prediction.

Needs to be real time not batch. The data needs to be processed as a stream since multiple sensors are used. pawan94 is right. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction

Here you go to the answer provided by google itself. I don't understand why would people use batch prediction when they its sensor data and online prediction is as well asynchronous. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#offline_batch_prediction:~:text=Predictive%20maintenance%3A%20asynchronously%20predicting%20whether%20a%20particular%20machine%20part%20will%20fail%20in%20the%20next%20N%20minutes%2C%20given%20the%20averages%20of%20the%20sensor%27s%20data%20in%20the%20past%2030%20minutes.

it refers to asincronou prediction I' go with C

D. I think we have to query data from the past 12 hours for the prediction, and that's the reason for exporting the data to Cloud Storage. Also, the predictions don't have to be real time.

Went with B

B. Online prediction, and need decoupling with Pub/Sub to make it asynchronous. Option A is synchronous.

Option C may not be the best choice for this use case because it involves using a batch prediction job in Vertex AI to perform scoring on preprocessed data. Batch prediction jobs are more suitable for scenarios where data is processed in batches, and results can be generated over a longer period, such as daily or weekly. In this use case, the requirement is to predict whether a machine part will fail in the next N minutes, given the average of each sensor's data from the past 12 hours. Therefore, real-time processing and prediction are necessary. Batch prediction jobs are not designed for real-time processing, and there may be a delay in receiving the predictions. Option B, on the other hand, is designed for real-time processing and prediction. The Pub/Sub and Dataflow components allow for real-time processing of incoming sensor data, and the trained ML model can be invoked for prediction in real-time. This makes it ideal for mission-critical applications where timely predictions are essential.

Its B, This architecture leverages the strengths of Pub/Sub, Dataflow, and Vertex AI. The system collects data from multiple sensors, which sends events to Pub/Sub. Pub/Sub can handle the high volume of incoming data and can buffer messages to prevent data loss. A Dataflow stream processing pipeline can consume the events in real-time and perform feature engineering and data preprocessing before invoking the trained ML model for prediction. The predictions are then sent to another Pub/Sub topic, where they can be consumed by a downstream system for monitoring. This architecture is highly scalable, resilient, and efficient, as it can handle large volumes of data and perform real-time processing and prediction. It also separates concerns by using a separate pipeline for data processing and another for prediction, making it easier to maintain and modify the system.

if you have sensors inyour architecture.. you need pub/sub...

B is most likely . if you search asynchronous on this page. it appears in the question wants to focus on online prediction with asynchronous mode. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#online_real-time_prediction and the question is the same as what has been explained in this section obviously. it is as below. Predictive maintenance: asynchronously predicting whether a particular machine part will fail in the next N minutes, given the averages of the sensor's data in the past 30 minutes. afte that, you can take a closer look at figure3 and read what it try to describle C and D it is the offline solution but you opt to use different tools. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#offline_batch_prediction

Asycnchromoue preciction = Batch prediction https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#offline_batch_prediction

B "Predictive maintenance: asynchronously predicting whether a particular machine part will fail in the next N minutes, given the averages of the sensor's data in the past 30 minutes." https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#offline_batch_prediction

Answer is B. https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#handling_dynamic_real-time_features

C, for me.

ref : https://cloud.google.com/architecture/minimizing-predictive-serving-latency-in-machine-learning#offline_batch_prediction