Use of /tmp for attribute passing not ideal

Description

In standard-ingest, the Validation processor requires the table schema attribute be written to a local file which is then passed as an argument. This pattern introduces another state management store which needs to be monitored and cleaned.

Ideally, the Nifi attributes should be automatically passed by the processor to the underlying spark job.

Realistically, another location such as HDFS should be used. This will reduce the number of stores used for ingestion/processing

Environment

None

Status

Assignee

Unassigned

Reporter

Azfar Aziz

Labels

Reviewer

None

Story point estimate

None

Components

Priority

Medium
Configure