Exam Certified Associate Developer for Apache Spark topic 1 question 35 discussion

E is the right option. See code below with Spark 3.5.1 # Summary statistics of a DataFrame from pyspark.sql import SparkSession from pyspark.sql.functions import col from pyspark.errors import PySparkTypeError spark = SparkSession.builder.appName("MyApp").getOrCreate() data = [ (0, 43161), (1, 51200), (2, None), (3, 78367), (4, None), ] storesDF = spark.createDataFrame(data, ["storeID", "sqft"]) try: storesDF.summary("mean").show() except Exception as e: print(e) try: storesDF.describe(all = True).show() except Exception as e: print(e) try: storesDF.describe("all").show() except Exception as e: print(e) try: storesDF.summary("all").show() except Exception as e: print(e) try: storesDF.describe().show() except Exception as e: print(e)

E is the correct one

E would be correct here

tested e is the right answer

E is the correct answer

check the documentation, mates. both methods receive names of columns as arguments, so E is correct!

E is correct, it's giving the output.

B is correct. On running the last option it gives error. TypeError: describe() got an unexpected keyword argument 'all'

The answer is B. Explanation: The describe() method in DataFrame returns a DataFrame with summary statistics for all numeric columns in the input DataFrame. By default, only the count, mean, standard deviation, minimum, and maximum values are returned, but additional statistics can be specified with the percentiles parameter. Setting the all parameter to True will include non-numeric columns in the output as well. Therefore, option B is the correct answer. Option A is not correct, as the summary() method only returns summary statistics for the specified column(s) and is not a valid option for returning summary statistics for all columns in the DataFrame. Option C is not correct, as the describe() method does not have an "all" option. Option D is also not correct, as the summary() method only returns summary statistics for the specified column(s) and does not have an "all" option. Option E is not incorrect, but it does not specify whether to include non-numeric columns in the output. Therefore, option B is a better answer.