Exam DP-700 topic 2 question 34 discussion

A. a hash function to compare the attributes in the source table.

The correct answer is: A. a hash function to compare the attributes in the source table. Here's why: Context: You are dealing with a Type 2 SCD table with 100 columns and 20 million rows. You need to detect changes to update the dimension appropriately, while minimizing resource consumption. Option A: A. a hash function to compare the attributes in the source table. This is the most efficient and scalable approach. You compute a hash (e.g., MD5, SHA-256) over a concatenation of all relevant (non-SCD metadata) columns in both the source and existing dimension table. By comparing the hashes, you avoid comparing each column directly. This minimizes CPU and memory usage, especially with many columns and large datasets. Commonly used in modern ETL/ELT tools for change detection in large datasets.

why don't use hash function on source side? the question lost many details. I just can guess they try to use incremental loading method. Normally we need keep Hask key values in target table column and use hash function to get changed imported rows from source side if their hash values cannot match with target hash value column.