Exam AWS Certified Machine Learning Engineer - Associate MLA-C01 topic 1 question 10 discussion

Amazon EMR with Spark is an excellent choice for aggregating, processing, and transforming large datasets from multiple sources (e.g., Amazon S3 and on-premises MySQL database). Spark jobs can handle both structured and unstructured. While Lake Formation is great for managing data lakes, it doesn’t provide the ETL and data processing capabilities required to aggregate and transform datasets from multiple sources.

Is it D? AWS Lake Formation ? EMR Spark jobs is more manual.

Lake Formation ain't only the storage service - it actually an umbrella over most AWS Glue Offerings. And cause it also provides fully serverless Spark, it seems to be better option than EMR. This point is quite tricky as often when someone refers to LF, they mean the governance part only.

The correct answer is D. AWS Lake Formation. Explanation: AWS Lake Formation is designed for aggregating, organizing, and securing large datasets from multiple sources (e.g., S3, on-premises databases). It simplifies the creation of a centralized data lake, enabling seamless integration and analysis of diverse data formats. This is particularly useful for tasks like fraud detection, where data comes from different sources. Amazon EMR Spark jobs (Option A) is more suited for large-scale data processing and analytics. While it can process and transform data, it requires more operational effort to configure and manage compared to AWS Lake Formation. Why AWS Lake Formation? Aggregates and organizes data from S3 and MySQL easily. Offers integrated data cataloging for better feature engineering. Reduces operational overhead compared to setting up EMR.

Lake Formation is the correct answer

AWS Lake Formation is a service designed to aggregate, catalog, and manage data from multiple data sources, including on-premises databases and Amazon S3, making it an ideal choice for this scenario. While Amazon EMR with Apache Spark is powerful for processing and analyzing large datasets, it focuses more on data processing than on data aggregation and cataloging. It doesn't inherently manage interdependencies or schema enforcement

Amazon EMR is correct answer for aggregation

Data Lake is used for data discovery

Once you specify where your existing databases are and provide your access credentials, Lake Formation reads the data and its metadata (schema) to understand the contents of the data source. It then imports the data to your new data lake and records the metadata in a central catalog. With Lake Formation, you can import data from MySQL, PostgreSQL, SQL Server, MariaDB, and Oracle databases running in Amazon RDS or hosted in Amazon EC2. Both bulk and incremental data loading are supported. https://docs.aws.amazon.com/lake-formation/latest/dg/what-is-lake-formation.html

While AWS Lake Formation could potentially be used in conjunction with other services for data lake management, Amazon EMR with Spark jobs is the most direct and powerful solution for aggregating and processing data from the various sources mentioned in this scenario. It provides the necessary tools to handle the data integration, address the class imbalance, and perform the complex feature engineering that may be required for the fraud detection model.

My first choice was Lake Formation

Yet another poorly worded AWS certification question. Here is my reasoning, the question is about "aggregate the data from S3 and on-premise mysql" and I do intend "aggregate" as put in the same place, therefore: A. No, while EMR spark job can connect to S3 and MySQL (spark can connect to mysql database), but it is a better tool to process data and then sore them in S3 B. No, KDS it is for delivering streaming data sources to specific destinations (S3, OpenSearch ...) C. No, DynamoDB is a nosql db that is not a great fit here D. Yes, Lake Formation "combine different types of structured and unstructured data into a centralized repository" https://docs.aws.amazon.com/lake-formation/latest/dg/what-is-lake-formation.html and "with Lake Formation, you can import your data using workflows" and as it is based on AWS Glue it supports both S3 and mysql

Question is which AWS service or feature can aggregate the data from the various data sources? So lake formation

I think it is A, it is more aligned with machine learning model

Lake formation can catalog data from various sources, it doesn't provide the data processing capabilities needed for this scenario. EMR is more appropriate in this case.

Data Aggregation: Lake Formation is designed to create a data lake, a centralized repository that stores and manages data from various sources, including S3, relational databases (like MySQL), and other data sources. Data Transformation: It can transform and clean data, making it suitable for analysis and machine learning. This includes handling class imbalance and feature interdependencies. Data Access: It provides a unified interface to access data, simplifying the process of integrating data from different sources into the ML model. While other options like Amazon EMR Spark jobs and Amazon Kinesis Data Streams could be used for data processing and streaming, they are not the most efficient and straightforward solutions for this specific use case. Amazon DynamoDB is a NoSQL database, not designed for batch data processing and aggregation. Therefore, AWS Lake Formation is the best choice to aggregate and prepare the data for the ML model. ref:https://docs.aws.amazon.com/lake-formation/latest/dg/what-is-lake-formation.html

Lake formation would be a better choice over EMR as it involes complexity setting up . For data aggregation and ETL processes, especially involving multiple data sources and ensuring data quality and security, AWS Lake Formation or Amazon Glue are more specialized and suitable option