Exam AZ-305 topic 2 question 22 discussion

The question mentions 'Data Lake Storage', not Delta Lake - there is no explicit indication that the data is stored in a delta lake format. Therefore I don't think that the Spark pool is needed. Nevertheless, Delta Lake is indeed a very confusing name for what is essentially a data format ("optimized storage layer").

Apache Spark pools in Azure Synapse enable data engineers to modify Delta Lake files Taken from: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/query-delta-lake-format

Agree. From what I can find SQL pool can't update delta lake files only Apache Spark can do that, assuming article is accurate below; https://www.jamesserra.com/archive/2022/03/azure-synapse-and-delta-lake/#:~:text=Serverless%20SQL%20pools%20do%20not%20support%20updating%20delta,in%20Azure%20Synapse%20Analytics%20to%20update%20Delta%20Lake.

How can you spent so much time to give explained answers, but you still get them wrong? First answer is correct, second one is Apache Spark pool. Serverless SQL pool doesn't provides updates: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/query-delta-lake-format. Do you see any information about updates there? Updates are possible in Apache Spark: https://docs.delta.io/latest/delta-update.html Btw - what "Apache Spark is best for big data analysis and ML tasks" have in common with Delta Lake updates? Are you copying the answers from the ChatGPT? I have worked with Databricks for 2 years and Apache Spark is the right answer. Apache Spark can be also used for small scenarios as it's not that expensive and is often used by data engineers, not just big data engineers

Requirement 1: Ingest data from Data Lake Storage into hash-distributed tables A dedicated SQL pool: This pool is specifically designed for high-performance data warehousing. It allows for the ingestion of large datasets into hash-distributed tables, optimizing performance and scalability. Hash distribution is a key feature of dedicated SQL pools to enhance query performance for large datasets. Recommendation: A dedicated SQL pool Requirement 2: Implement, query, and update data in Delta Lake A serverless Apache Spark pool: Delta Lake is an open-source storage layer that brings ACID transactions to Apache Spark. It is optimized for big data workloads and is best utilized with Apache Spark pools. The serverless Apache Spark pool in Azure Synapse provides a managed Spark environment, ideal for working with Delta Lake for querying, updating, and managing large datasets.