How to light your 'Spark on a stick'

Launch the Spark Scala shell

To run the Spark Scala shell, go back to the Windows Explorer window that shows the Spark USB drive, navigate into the spark subdirectory, then hold down the shift button while you right-click on bin. From the drop-down menu, choose Open command window here:


Now you should see a CMD line window displaying the USB:\spark\bin> directory at the prompt:


Type in spark-shell and hit enter to launch the Scala spark shell:

USB:\spark\bin>spark-shell
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.1.0
      /_/

Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_67)
Type in expressions to have them evaluated.
Type :help for more information.
Spark context available as sc.

scala>


Try running the sc.master command at the shell to verify that the "Spark shell" application is running in local mode:

scala> sc.master
res2: String = local[*]

The asterisk in brackets means to run Spark locally with as many worker threads as logical cores on your machine.



You may get a Windows Firewall popup when the shell loads for the first time. At the Firewall window, select both Private and Public networks and click Allow: