Spark Shell not able to run in cluster mode in 0.9.1 sandboxes

Description

The 0.9.1 sandboxes seem unable to support spark shell cluster mode. I'm not sure why yet. My issue came up while attempting to test user impersonation in the VirtualBox sandbox with my local mods. After some time I decided to test the 0.9.1 CDH and HDP sandboxes. For CDH I loaded a pre-existing AWS sandbox of mine that I believe had no modifications from the base sandbox and I noticed the error there. So I went to EC2 and created a fresh 0.9.1 HDP sandbox. After waiting some time for start up I modified spark.properties to be:

I then went to Visual Query page and waited until I saw a process appear:

I watched the spark shell logs proceed through the steps of launching a spark job until it fails with:

Environment

None

Assignee

Unassigned

Reporter

Tim Harsch

Labels

None

Reviewer

None

Story point estimate

None

Components

Affects versions

Priority

Medium
Configure