Locating the logs

In local mode, Spark does not log externally to a log file.

Instead logs are just displayed at the CMD-line console.

By default the logging level is Warnings and above (so errors and warnings will be displayed).

There are various logging levels in Log4j:

• ALL: turns on all logging
• OFF: turns off all logging
• INFO: shows info, warnings and errors
• WARN: shows warnings and errors
• ERROR: shows just errors
• DEBUG: shows fine-grained info events that are useful for debugging an application
• TRACE: shows even more fine-grained info events than DEBUG

If you want to turn the logging level up so that even Info messages are displayed, you can change line #2 from WARN to INFO in the file USB:\spark\conf\log4j.properties

Leave the logging level at the default of WARN for now and continue.