Exam Professional Data Engineer topic 1 question 308 discussion

- It addresses the likely issue: that the signed URLs have expired or are otherwise invalid. By creating a new TSV file with freshly generated signed URLs (with a longer validity period), you're ensuring that the Storage Transfer Service has valid authorization to access the files. - Splitting the TSV file and running parallel jobs might help in managing the workload more efficiently and overcoming any limitations related to the number of files or transfer speed.

C. Create a new TSV file for the remaining files by generating signed URLs with a longer validity period. Split the TSV file into multiple smaller files and submit them as separate Storage Transfer Service jobs in parallel. Here's why: HTTP 403 errors: These errors indicate unauthorized access, but since you verified the source system and signed URLs, the issue likely lies with expired signed URLs. Renewing the URLs with a longer validity period prevents this issue for the remaining files. Separate jobs: Splitting the file into smaller chunks and submitting them as separate jobs improves parallelism and potentially speeds up the transfer process. Avoid manual intervention: Options A and D require manual intervention and complex setups, which are less efficient and might introduce risks. Longer validity: While option B addresses expired URLs, splitting the file offers additional benefits for faster migration.

C. Create a new TSV file for the remaining files by generating signed URLs with a longer validity period. Split the TSV file into multiple smaller files and submit them as separate Storage Transfer Service jobs in parallel. The HTTP 403 errors likely occurred because the signed URLs expired before the transfer could complete. By generating new signed URLs with a longer validity period, you can ensure that the URLs remain valid for the duration of the transfer. Splitting the TSV file into smaller files and submitting them as separate jobs can help manage the load and reduce the risk of timeouts or other issues

got this one on the exam, aug 2024, passed

Option C - agree with Raaad

C. Create a new TSV file for the remaining files by generating signed URLs with a longer validity period. Split the TSV file into multiple smaller files and submit them as separate Storage Transfer Service jobs in parallel.