Simple steps to run Spark 1.5.2 on Ubuntu 14.04 with Yarn (Hadoop 2.7.1)
Install Hadoop (see http://dmitrypukhov.pro/install-hadoop-on-ubuntu/)
Download pre-build Spark from http://spark.apache.org/downloads.html and unpack, say to /opt/spark/Edit spark-env.sh: Copy /opt/spark/conf/spark-env.sh.template to /opt/spark/conf/spark-env.sh, edit created spark-env.sh and add the following content there:
|
1 2 |
HADOOP_HOME=<your hadoop home dir> SPARK_DIST_CLASSPATH=<output of your "hadoop classpath" command> |
Edit ~/.bashrc file, add:
|
1 |
export PATH=$PATH:/opt/spark/bin |
That’s all. Restart terminal to load last changes, type spark-shell in command line and enjoy 🙂
