How to light your 'Spark on a stick'

Locating the logs

In local mode, Spark does not log externally to a log file.

Instead logs are just displayed at the CMD-line console.

By default the logging level is Warnings and above (so errors and warnings will be displayed).

There are various logging levels in Log4j:

  • ALL: turns on all logging
  • OFF: turns off all logging
  • INFO: shows info, warnings and errors
  • WARN: shows warnings and errors
  • ERROR: shows just errors
  • DEBUG: shows fine-grained info events that are useful for debugging an application
  • TRACE: shows even more fine-grained info events than DEBUG

If you want to turn the logging level up so that even Info messages are displayed, you can change line #2 from WARN to INFO in the file USB:\spark\conf\

Leave the logging level at the default of WARN for now and continue.