Transform Template Missing HDFS Folders

Description

The Spark "Execute Script" step in the transformation template is failing in the MapR 6.0.1 environment. This is because two HDFS folders are not created during initialization.

Caused by: org.apache.hadoop.security.AccessControlException: User nifi(user id 995) has been denied access to create .hive-staging_hive_2018-08-15_22-28-38_315_1093715275587574699-1

mkdirs failed for /model.db/users/transform_test_1/feed/.hive-staging_hive_2018-0, error 13

The fix is to add the two below properties to the CreateHDFS Folder processor

${hive.ingest.root}/${category}/${feed}/feed
${hive.ingest.root}/${category}/${feed}/invalid

Environment

None

Assignee

Jeremy Merrifield

Reporter

Jeremy Merrifield

Labels

None

Reviewer

None

Story point estimate

None

Story Points

1

Sprint

None

Fix versions

Affects versions

Priority

Medium
Configure