I have my data in parquet format and want to load and query it using Spark SQL.
Start Spark shell
|
1 |
spark-shell |
Load parquet folder to table
|
1 2 |
val myDataFrame=sqlContext.load("s3://my-bucket/my-parquet-folder/") myDataFrame.registerTempTable("myTable") |
Now we can use this table for SQL queries:
|
1 |
sqlContext.sql("select * from myTable").first() |
