How to get JSON attribute and use it in your Spark program?

Here is the scala code to fetch each attribute from JSON and use it in your code jsonFile = s”hdfs://${jsonFilefolder}”……… //If present in HDFS else provide local directory path val strRdd = sc.parallelize(List(sc.textFile(jsonFile).collect().mkString(” “))) …..//Create JSONRDD val jsonDF = ssc.jsonRDD(strRdd) jsonDF.printSchema() ………//print schema of JSON data val root = jsonDF.select(“RootAttribute”).collect().mkString(“”).replaceAll(“\\[“, “”).replaceAll(“\\]”, “”)……….//Replace all [,comma,] to […]

Continue reading