Exam Certified Associate Developer for Apache Spark topic 1 question 29 discussion

Correct anwser is A. even for most updated version, spark 3.4. na.fill() still functioning, it is an alias of fillna() Mr. 4be8126 , 你可真是张嘴就来啊

The most correct answer seems to be A: Code below with Spark 3.5.1. ```python from pyspark.sql import SparkSession from pyspark.sql.functions import col from pyspark.errors import PySparkTypeError spark = SparkSession.builder.appName("MyApp").getOrCreate() data = [ (0, 43161), (1, 51200), (2, None), (3, 78367), (4, None), ] storesDF = spark.createDataFrame(data, ["storeID", "sqft"]) # storesDF.show() try: storesDF.na.fill(30000, col("sqft")) except PySparkTypeError as e: print(e) storesDF.na.fill(30000, "sqft").show() storesDF.na.fill(30000, ["sqft"]).show() storesDF.fillna(30000, ["sqft"]).show() storesDF.fillna(30000, "sqft").show() ```

We don't need any replacement here. A would be correct. In PySpark both fillna() and fill() are used to replace missing or null values of a DataFrame. Functionally they both perform same. One can choose either of these based on preference. These are used mainly for handling missing data in PySpark.

The correct answer is either A or E, depending on the version of Spark being used. In Spark 2.x, the correct method to replace missing values is na.fill(). Option A is correct in Spark 2.x, as it correctly specifies the column to apply the fill operation to using a Column object. However, in Spark 3.x, the method has been renamed to fillna(). Therefore, in Spark 3.x, the correct answer is E, as it uses the correct method name. Both A and E accomplish the same task of replacing missing values in the sqft column with 30,000, so either answer can be considered correct depending on the version of Spark being used.

the answer should be A. See this link for reference https://sparkbyexamples.com/pyspark/pyspark-fillna-fill-replace-null-values/

The error in the code block is that the method na.fill() should be replaced by fillna() to fill the missing values in the column "sqft" with the value 30,000.