You are managing a Cloud Dataproc cluster. You need to make a job run faster while minimizing costs, without losing work in progress on your clusters. What should you do?
A.
Increase the cluster size with more non-preemptible workers.
B.
Increase the cluster size with preemptible worker nodes, and configure them to forcefully decommission.
C.
Increase the cluster size with preemptible worker nodes, and use Cloud Stackdriver to trigger a script to preserve work.
D.
Increase the cluster size with preemptible worker nodes, and configure them to use graceful decommissioning.
graceful decommissioning: to finish work in progress on a worker before it is removed from the Cloud Dataproc cluster.
https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scaling-clusters
All your workers need to be the same kind. Use Graceful Decommissioning for don't lose any data and add more(increase the cluster) preemptible workers because there are more cost-effective .
A. "graceful decommissioning" is not a configuration value but a parameter passed with scale down action - to decrease the number of workers to save money (see Graceful Decommissioning as an option to use when downsizing a cluster to avoid losing work in progress)
https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/scaling-clusters
Why scale a Dataproc cluster?
to increase the number of workers to make a job run faster
to decrease the number of workers to save money (see Graceful Decommissioning as an option to use when downsizing a cluster to avoid losing work in progress).
to increase the number of nodes to expand available Hadoop Distributed Filesystem (HDFS) storage
All your workers need to be the same kind. Use Graceful Decommissioning for don't lose any data and add more preemptible workers because there are more cost-effective
A voting comment increases the vote count for the chosen answer by one.
Upvoting a comment with a selected answer will also increase the vote count towards that answer by one.
So if you see a comment that you already agree with, you can upvote it instead of posting a new comment.
AzureDP900
Highly Voted 1 year, 4 months agorocky48
Most Recent 4 months, 3 weeks agoPrakzz
1 year, 4 months agowan2three
1 year, 3 months agoyafsong
1 year, 4 months agoodacir
1 year, 4 months agoskp57
1 year, 5 months agoJohn_Pongthorn
1 year, 7 months agohauhau
1 year, 6 months agoodacir
1 year, 4 months agoAWSandeep
1 year, 7 months ago