Question : dataframe column json parser spark scala
Answered by : ogun-ozalp
import org.apache.spark.sql.functions.{lit, schema_of_json, from_json}
import collection.JavaConverters._
val schema = schema_of_json(lit(df.select($"jsonData").as[String].first))
df.withColumn("jsonData", from_json($"jsonData", schema, Map[String, String]().asJava))Source : https://stackoverflow.com/questions/34069282/how-to-query-json-data-column-using-spark-dataframes | Last Update : Thu, 19 May 22