Exam Certified Data Engineer Associate topic 1 question 67 discussion

correct answer: D explanation: The MERGE INTO command is used when you need to perform both insertions and updates (or deletes) in one operation based on whether a match exists. It is particularly useful for maintaining up-to-date data and ensuring there are no duplicate records in the target table. This is often referred to as an "upsert" operation (update + insert). When the target table needs to be kept free of duplicate records, and there's a need to update existing records or insert new ones based on some matching condition, MERGE INTO is the appropriate command. The INSERT INTO command, on the other hand, is used to add new records to a table without regard for whether they duplicate existing records. Options A, B, C, and E do not specifically require the use of MERGE INTO. Therefore, D is the correct answer.

MERGE INTO you can upsert (update insert) data from a source table, view or dataframe into the target table. Merge operation allows updates, insets and deletes to be completed in a single atomic transaction. The main benefit of using the MERGE INTO is to avoid duplicates but does not inherently remove duplicates.

D is answer here

D is correct

Ans D : With merge , you can avoid inserting the duplicate records. The dataset containing the new logs needs to be deduplicated within itself. By the SQL semantics of merge, it matches and deduplicates the new data with the existing data in the table, but if there is duplicate data within the new dataset, it is inserted. https://docs.databricks.com/en/delta/merge.html#:~:text=With%20merge%20%2C%20you%20can%20avoid%20inserting%20the%20duplicate%20records.&text=The%20dataset%20containing%20the%20new,new%20dataset%2C%20it%20is%20inserted.