Exam Certified Data Engineer Associate topic 1 question 19 discussion

In Databricks, the COPY INTO command is designed to prevent duplicate data ingestion. When files are copied into a table, Databricks keeps track of the files that have already been processed using a file log. If a file has already been copied, subsequent runs of the COPY INTO command will skip that file to avoid duplication.

Just got 100% on the test. C was correct.

Correct answer is C

C. The previous day’s file has already been copied into the table. The COPY INTO statement is generally used to copy data from files or a location into a table. If the data engineer runs this statement daily to copy the previous day’s sales into the "transactions" table and the number of records hasn't changed after today's execution, it's possible that the data from today's file might not have differed from the data already present in the table. If the files in the "/transactions/raw" location are expected to contain distinct data for each day and the number of records in the table remains the same, it implies that the data engineer might have already copied today's data previously, or today's data was identical to the data already present in the table. Options A, B, D, and E don't accurately explain why the statement might not have copied new records into the table based on the provided scenario.

C is correct

If the table "transaction" is an external table, then option E, if its internal C should suffice.

COPY INTO statement does skip already copied rows.

C is correct ans

E is the correct answer, because immediately after using copy into you might query the cashed version of the table.

https://docs.databricks.com/en/ingestion/copy-into/index.html The COPY INTO SQL command lets you load data from a file location into a Delta table. This is a re-triable and idempotent operation; files in the source location that have already been loaded are skipped. if there are no new records, the only consistent choice is C no new files were loaded because already loaded files were skipped.

C The COPY INTO statement copies the data from the specified files into the target table. If the previous day's file has already been copied into the table, then the COPY INTO statement will not copy any new records into the table.

COPY INTO Loads data from a file location into a Delta table. This is a retriable and idempotent operation—files in the source location that have already been loaded are skipped.

Answer: B FILES = ('f1.json', 'f2.json', 'f3.json', 'f4.json', 'f5.json') https://docs.databricks.com/ingestion/copy-into/examples.html

C- https://docs.databricks.com/ingestion/copy-into/tutorial-notebook.html Because this action is idempotent, you can run it multiple times but data will only be loaded once.

Option C is the correct answer.

i am not sure whether C is the correct answer, but A is definitely not right

option C