Exam AWS Certified Data Engineer - Associate DEA-C01 topic 1 question 21 discussion

How does Data Quality obfuscate PII? You can do this directly in Glue Studio: https://docs.aws.amazon.com/glue/latest/dg/detect-PII.html

Option C involves additional steps and complexity with creating rules in AWS Glue Data Quality, which adds more operational effort compared to directly using AWS Glue Studio's capabilities.

Actually it is B. No need to create a rule in AWS Glue.

B. Use the Detect PII transform in AWS Glue Studio to identify the PII. Obfuscate the PII. Use an AWS Step Functions state machine to orchestrate a data pipeline to ingest the data into the S3 data lake. Detect PII transform only detects. Obfuscate the PII ok but how ? Answer C explain how

Why C is better than B: Obfuscation clarity: Option C explicitly mentions using a Glue Data Quality rule to obfuscate PII, while option B does not specify how obfuscation is implemented. Accuracy: Glue Data Quality provides a more structured way to handle obfuscation compared to relying solely on Glue Studio's PII detection. Thus, C is the most accurate and operationally efficient solution.

The keyt

B provides a streamlined, mostly visual approach using purpose-built tools for data processing and PII handling, making it the solution with the least operational effort.

https://aws.amazon.com/blogs/big-data/automated-data-governance-with-aws-glue-data-quality-sensitive-data-detection-and-aws-lake-formation/

https://docs.aws.amazon.com/glue/latest/dg/detect-PII.html

anwser is C

I don't think we need to use much more services to fulfill these requirements. Just AWS Glue is enough, it can detect and obfuscate PII data already. Source: https://docs.aws.amazon.com/glue/latest/dg/detect-PII.html#choose-action-pii

We cannot directly handle PII with Glue Studio, and Glue Data Quality can be used to handle PII.

A very easy was is to use the SDK to identify PII. https://docs.aws.amazon.com/code-library/latest/ug/comprehend_example_comprehend_DetectPiiEntities_section.html

The transform Detect PII in AWS Glue Studio is specifically used to identify personally identifiable information (PII) within the data. It can detect and flag this information, but on its own, it does not perform the obfuscation or removal of these details. To effectively obfuscate or alter the identified PII, an additional transformation would be necessary. This could be accomplished in several ways, such as: Writing a custom script within the same AWS Glue job using Python or Scala to modify the PII data as needed. Using AWS Glue Data Quality, if available, to create rules that automatically obfuscate or modify the data identified as PII. AWS Glue Data Quality is a newer tool that helps improve data quality through rules and transformations, but whether it's needed will depend on the functionality's availability and the specificity of the obfuscation requirements

Answer is option C. Period

B is correct. C: glue data quality cannot obfuscate the PII D: need to write code but the question is the "LEAST operational effort"

In python --- from awsglue.utils import getResolvedOptions from pyspark.context import SparkContext from awsglue.context import GlueContext from pyspark.sql import SparkSession # Initialize Spark session spark = SparkSession.builder \ .appName("Example Glue Job") \ .getOrCreate() # Initialize Glue context glueContext = GlueContext(SparkContext.getOrCreate()) # Retrieve Glue job arguments args = getResolvedOptions(sys.argv, ['JOB_NAME']) # Define your EMR step emr_step = [ { "Name": "My EMR Step", "ActionOnFailure": "CONTINUE", "HadoopJarStep": { "Jar": "s3://your-bucket/emr-scripts/your_script.jar", "Args": [ "arg1", "arg2" ] } } ] # Execute the EMR step response = glueContext.start_job_run(args['JOB_NAME'], job_run_args={'--extra-py-files': 'your_script.py'}) print(response)