When using AWS EMR, SparkLauncher does not set spark env
Description
It appears that when the data transformation template is used, ExecuteSparkJob in the reusable template fails to set the env using SparkLauncher (ExecuteSparkJob.java:518).
Reproducible steps using the Kylo Sandbox AMI and AWS EMR can be found attached and on the community forum.
This presents the error "No encryption/decryption key has been configured - please see configuration documentation" in the spark logs.
Environment
None
Activity
Show:
Gibran Akram
May 29, 2019, 12:01 PM
Did you alter the processor code for this workaround?
This blocks us from doing any transformation successfully.