Validator app finishes successfully but Yarn marks it as failed.

Description

On a cloudera cluster 5.12.0 and using spark version 2.2.0.cloudera1, when the app is launched on yarn-cluster mode it finishes successfully but Yarn marks it as failed.

From some researches it looks like there is a System.exit(0) call at the end of the app: https://stackoverflow.com/questions/39327616/why-is-spark-applications-final-status-failed-while-it-finishes-successfully


The Validator job (and probably the Profiler job) should do the following, instead of calling System.exit()

Environment

None

Assignee

Greg Hart

Reporter

Fabian Necci

Labels

None

Reviewer

None

Story point estimate

None

Components

Sprint

None

Fix versions

Affects versions

Priority

Medium
Configure