Exam Certified Associate Developer for Apache Spark topic 1 question 124 discussion

It's D

I believe D is the correct one according to documentation from Databricks [1]: "The driver process runs your main() function, sits on a node in the cluster, and is responsible for three things: maintaining information about the Spark Application; responding to a user’s program or input; and analyzing, distributing, and scheduling work across the executors (defined momentarily)." Addittionaly: "The cluster manager controls physical machines and allocates resources to Spark Applications." Based on the above we could say that cluster manager is charge assign resources (CPU, Memory, etc) to the VMs used. Keep in mind that this is based on the definition from Databricks other definitions may include what was mentioned by cookiemonster42. [1] https://www.databricks.com/glossary/what-are-spark-applications

Should be B - D - Spark driver is not directly responsible for scheduling the execution of data by various worker nodes in cluster mode. It submits tasks to the cluster manager (e.g., YARN, Mesos, or Kubernetes), and the cluster manager handles the scheduling of tasks on worker nodes.