Exam Certified Associate Developer for Apache Spark topic 1 question 41 discussion

Answer: A See code example below with Spark 3.5.1: from pyspark.sql import SparkSession spark = SparkSession.builder.appName("MyApp").getOrCreate() data = [ (0, 43161, "A"), (1, 51200, "A"), (3, 78367, "B"), ] storesDF = spark.createDataFrame(data, ["storeID", "sqft", "division"]) def assess_performance(x): return "Large" if x > 50000 else "Small" spark.udf.register("ASSESS_PERFORMANCE", assess_performance, "STRING") storesDF.createOrReplaceTempView("stores") df = spark.sql("SELECT StoreID, ASSESS_PERFORMANCE(sqft) AS performance FROM stores") df.show()

def assessperformance(): return 'Good' spark.udf.register("assessperformance",assessperformance) df = spark.sql("SELECT assessperformance()") df.show() A

Answer: A Explanation: udf: create a user-defined function (UDF) in PySpark register: register the UDF with Spark so it can be used in SQL queries "ASSESS_PERFORMANCE": name the UDF "ASSESS_PERFORMANCE" assessPerformance: specify the Python function to use for the UDF ASSESS_PERFORMANCE: use the registered UDF in the SQL query to apply the assessPerformance() function to the customerSatisfaction column.