Exam Certified Data Engineer Professional topic 1 question 52 discussion

It's B, there is no column with that name

E is correct

B is correct. I created a simple df and ran this code : display(df.select(3*"heartrate")) and I got this error: AnalysisException: [UNRESOLVED_COLUMN.WITH_SUGGESTION] A column or function parameter with name `heartrateheartrateheartrate` cannot be resolved. Did you mean one of the following? [`heartrate`, `device_id`, `time`, `mrn`].; 'Project ['heartrateheartrateheartrate] +- LogicalRDD [device_id#2L, heartrate#3L, mrn#4L, time#5], false

from pyspark.sql import SparkSession from pyspark.sql.functions import col # Initialize Spark session spark = SparkSession.builder.appName("ErrorReproduction").getOrCreate() # Create a sample DataFrame with similar structure data = [ (1, 72, 12345, "2023-01-01 10:00:00"), (2, 68, 67890, "2023-01-01 10:01:00"), (3, 75, 54321, "2023-01-01 10:02:00") ] columns = ["device_id", "heartrate", "mrn", "time"] df = spark.createDataFrame(data, columns) # This will produce the AnalysisException # The error occurs because you're trying to multiply a string "heartrate" by 3 literally # instead of referencing the column and multiplying its values display(df.select(3*"heartrate")) # display(df.select(3*col("heartrate")))

It's B. Regarding E, a syntax error would mean that the query is not valid due to a wrongfully written SQL statement. However, this is not the case. The column just does not exist.

https://sparkbyexamples.com/spark/spark-cannot-resolve-given-input-columns/

the answer is E, because df.select(3*df['heartrate']).show() perfectly returns

Answer is E df.select(3*df['heartrate']) returns perfect result without error

Answer B

Answer is B

No such column found