A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source. That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months.
Which describes how Delta Lake can help to avoid data loss of this nature in the future?
imatheushenrique
5 months, 3 weeks agoojudz08
9 months, 2 weeks agoDAN_H
9 months, 4 weeks agokz_data
10 months, 2 weeks agoalexvno
1 year ago