Exam AWS Certified Solutions Architect - Associate SAA-C03 topic 1 question 103 discussion

This is the purpose of bookmarks: "AWS Glue tracks data that has already been processed during a previous run of an ETL job by persisting state information from the job run. This persisted state information is called a job bookmark. Job bookmarks help AWS Glue maintain state information and prevent the reprocessing of old data." https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html

A. Job bookmarks in Glue allow you to track the last-processed data in a job. By enabling job bookmarks, Glue keeps track of the processed data and automatically resumes processing from where it left off in subsequent job runs. B. Results in the permanent removal of the data from the S3, making it unavailable for future job runs. This is not desirable if the data needs to be retained or used for subsequent analysis. C.It would only affect the parallelism of the job but would not address the issue of reprocessing old data. It does not provide a mechanism to track the processed data or skip already processed data. D. It is not directly related to preventing Glue from reprocessing old data. The FindMatches transform is used for identifying and matching duplicate or matching records in a dataset. While it can be used in data processing pipelines, it does not address the specific requirement of avoiding reprocessing old data in this scenario.

B: Glue can delete DataSet but this option is too vague to consider or too open to mean anything C: Won't help with repeated ETL. This property affects parallelism D: Too vague

https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html

The best solution is to edit the AWS Glue job to use job bookmarks. Job bookmarks allow AWS Glue ETL jobs to track which data has already been processed during previous runs. This prevents reprocessing of old data. Deleting the data after processing would cause the data to be lost and unavailable for future processing. Reducing the number of workers may improve performance but does not prevent reprocessing of old data. Using a FindMatches ML transform is used for record matching, not preventing reprocessing. So the solutions architect should enable job bookmarks in the AWS Glue job configuration. This will allow the ETL job to keep track of processed data and only transform the new data added since the last run.

Job bookmark to make sure that the glue job will not process already processed files.

Job bookmarks are used in AWS Glue ETL jobs to keep track of the data that has already been processed in a previous job run. With bookmarks enabled, AWS Glue will read the bookmark information from the previous job run and will only process the new data that has been added to the data source since the last job run. This saves time and reduces costs by eliminating the need to reprocess old data. Therefore, a solutions architect should edit the AWS Glue ETL job to use job bookmarks so that it will only process new data added to the S3 bucket since the last job run.

Job bookmarks enable AWS Glue to track the data that has been processed in a previous run of the job. With job bookmarks enabled, AWS Glue will only process new data that has been added to the S3 bucket since the previous run of the job, rather than reprocessing all data every time the job runs.

Delete files in S3 freely is not good. so B is not correct,

A is correct

Option A. Edit the job to use job bookmarks. Job bookmarks in AWS Glue allow the ETL job to track the data that has been processed and to skip data that has already been processed. This can prevent AWS Glue from reprocessing old data and can improve the performance of the ETL job by only processing new data. To use job bookmarks, the solutions architect can edit the job and set the "Use job bookmark" option to "True". The ETL job will then use the job bookmark to track the data that has been processed and skip data that has already been processed in subsequent runs.

Option A

It's obviously A. Bookmarks serve this purpose

A is correct

A https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html