Exam Professional Data Engineer topic 1 question 274 discussion

- New BigQuery Table with CMEK: This option involves creating a new BigQuery table configured to use a CMEK from Cloud KMS. It directly addresses the need to use a CMEK for data at rest in BigQuery. - Migrate Data: Migrating data from the old table (encrypted with a Google-managed key) to the new table (encrypted with CMEK) ensures that all existing data complies with the new policy.

should be B. Pub/Sub is not designed for storing data at rest.

Agree with raaad

Requirement for Cloud KMS keys: The new organization policy requires using keys from a centralized Cloud KMS project for encrypting data at rest. This necessitates the use of customer-managed encryption keys (CMEK). BigQuery table encryption: The existing BigQuery table is encrypted with a Google-managed key. To meet the new policy, a new table needs to be created with CMEK. Pub/Sub topic encryption: Since the data is ingested directly from a Pub/Sub subscription, the Pub/Sub topic also needs to use CMEK to ensure end-to-end encryption with customer-managed keys. Data migration: The existing data in the old BigQuery table needs to be migrated to the new CMEK-encrypted table to ensure all data complies with the new policy

"The best solution here is B. Create a new BigQuery table by using customer-managed encryption keys (CMEK), and migrate the data from the old BigQuery table. Here's why: Customer-Managed Encryption Keys (CMEK): CMEKs allow you to have granular control over your encryption keys, complying with the organization's policy to use keys from a centralized Cloud KMS project. Data Migration: Since the data in the existing table is already encrypted with a Google-managed key, you cannot retroactively change the encryption key for that table. Migrating the data to a new table with the correct encryption is the most efficient way to meet compliance.

D. Create a new BigQuery table and Pub/Sub topic by using customer-managed encryption keys (CMEK), and migrate the data from the old BigQuery table. This approach comprehensively addresses the requirement to use CMEK from a centralized Cloud KMS project for encrypting data at rest: Create a new Pub/Sub topic configured to use CMEK from the centralized Cloud KMS project. Create a new BigQuery table with CMEK enabled, using the same centralized Cloud KMS project. Update the ingestion process to use the new Pub/Sub topic to feed data into the new BigQuery table. Migrate existing data from the old BigQuery table to the new BigQuery table to ensure all data complies with the new encryption policy.

B, been there, done that...

BigQuery and Pub/Sub shall be encrypted using CMEK using new versions of each one. https://cloud.google.com/pubsub/docs/encryption#using-cmek

Data at rest in requirement = Big Query ONLY. Pub/Sub is data in movement - overkill for the solution

D- BigQuery and Pub/sub are automatically encrypted but here we need to apply a more secured policy by using CMEK so we need to use it for bigquery and pub/sub to meet this policy

B. Create a new BigQuery table by using customer-managed encryption keys (CMEK), and migrate the data from the old BigQuery table. Most Voted

BigQuery allows you to encrypt data at rest using either Google-managed encryption keys or customer-managed encryption keys (CMEK) from Cloud KMS. Since the new policy requires using keys from a centralized Cloud KMS project, you need to create a new BigQuery table that is configured to use CMEK for encryption. After creating the new table with CMEK, you can migrate the data from the old table (encrypted with Google-managed keys) to the new table (encrypted with CMEK). This approach ensures that the data in the BigQuery table is encrypted using the required CMEK while preserving the existing data. Creating a new BigQuery table and Pub/Sub topic with CMEK is not necessary because the focus is on encrypting the data at rest in BigQuery. The existing Pub/Sub subscription can still be used to ingest data into the new BigQuery table.

D - 'as new organization policy that requires you to use keys from a centralized Cloud Key Management Service (Cloud KMS) project to encrypt data at rest.' Therefore, the Pub/Sub default Google-managed encryption key is not sufficient as the organization requires it's own CMEK that is to be generated from a centralized Cloud KMS project.

Agree with ML6 and Smakyel. To encrypt data at rest we should encrypt the data in PubSub and BigQuery

Only option D complies with the organisation policy: - By creating a new Pub/Sub topic with customer-managed encryption keys (CMEK), any new data ingested into Pub/Sub will be encrypted with the (!) organization's desired encryption keys (!). - Creating a new BigQuery table with CMEK ensures that all data stored in BigQuery, both newly ingested and migrated historical data, is encrypted according to organizational policies. - Migrating the data from the old BigQuery table to the new one ensures that historical data is also encrypted with the new keys, thus meeting the organization's requirements for encryption at rest for both Pub/Sub and BigQuery.

Option D - I get the discussion about B and D, but also pub/sub has some data at rest, e.g. messages with retention period. To comply with the organisation policy, we need to adapt also pub/sub

D. We should use new CMSK for both pubsub topic and BQ tables along with migrating old data.