Exam Certified Data Engineer Professional topic 1 question 146 discussion

Actual exam question from Databricks's Certified Data Engineer Professional

Question #: 146
Topic #: 1

[All Certified Data Engineer Professional Questions]

A data engineer is configuring a pipeline that will potentially see late-arriving, duplicate records.

In addition to de-duplicating records within the batch, which of the following approaches allows the data engineer to deduplicate data against previously processed records as it is inserted into a Delta table?

A. Rely on Delta Lake schema enforcement to prevent duplicate records.
B. VACUUM the Delta table after each batch completes.
C. Perform an insert-only merge with a matching condition on a unique key.
D. Perform a full outer join on a unique key and overwrite existing data.

Show Suggested Answer

Suggested Answer: C 🗳️

by m79590530 at Oct. 20, 2024, 4:54 p.m.

Comments

Submit Cancel

KadELbied

2 months ago

Selected Answer: C

Suretly C

upvoted 1 times

...

benni_ale

8 months, 1 week ago

Selected Answer: C

C seems logical

upvoted 1 times

...

m79590530

8 months, 3 weeks ago

Selected Answer: C

From all the provided options Answer C is the only meaningful and possible one. Also MERGE INTO ... WHEN NOT MATCHED INSERT *; is a standard solution for adding/appending non-existing records (by key) to the target table withOUT duplicating.

upvoted 2 times

...