Unsupported Snappy codec in Spark Yarn-cluster mode in CDH

Description

Setting up Kylo Spark Shell to run in yarn-cluster deploy mode, or setting the ExecuteSparkJob NiFi processor to run in yarn-cluster deploy mode, will trigger an exception because the Snappy native libraries are not passed/loaded correctly.
This only manifests in Cloudera environment.

Attached is the spark job log, taken from yarn.

Workaround is to disable Snappy, and use LZ4
Current workaround for Kylo:

Current workaround for NiFi:

Discussion between our client and Cloudera support:

Environment

Cloudera 5.10+

Assignee

Unassigned

Reporter

Claudiu Stanciu

Labels

None

Reviewer

None

Story point estimate

None

Components

Affects versions

Priority

High
Configure