Kylo NiFi processors should gracefully handle when Kylo is temporarily down

Description

We need to make sure that NiFi processors that access Kylo yield or penalize flow files when it is determined that Kylo is currently down. This is so that processing can resume once Kylo is back up.

The metadata controller service will have to be able to detect when an access failure is due to Kylo unavailable vs some other HTTP error, such as a non-existent water mark due to user configuration error.

Affected components:

  • Metadata Controller Service

  • LoadHighWaterMark

  • ReleaseHighWaterMark

  • InitializeFeed

We need to be careful with water marks to make sure we don't cancel a water mark that should be canceled.

Environment

None

Status

Assignee

Sean Felten

Reporter

Sean Felten

Labels

None

Reviewer

None

Story point estimate

None

Components

Priority

High
Configure