Invalidate InitializeFeed cache when feed deleted (PR #90)

Description

Re-produce steps:
1. create a 'Data Transformation' feed named foo
2. run the feed 'foo' and wait for it to complete
3. delete the feed 'foo'
4. recreate the feed with the same category and feed names
5. run the feed again

Expected result:
The feed should completed

Actual result:
The feed failed with error says that the 'foo_valid' table doesn't exist

The root cause is that the 'InitializeFeed' processor save the old feed ID in its cache (i.e. FeedProcessor.feedIdCache) and its 'InitializationStatus' in another cache,
but when deleting the feed 'foo', both the feed ID and its 'InitializationStatus' are not removed from the caches.

So when the feed with the same name was created and run, the InitializeFeed processor will get the feed ID of the deleted feed and its 'InitializationStatus' from the caches instead of the ones of the new feed and use it in the following computing.

Environment

None

Activity

Show:
Boying Lu
August 10, 2018, 6:25 AM

The PR against this issue is available at https://github.com/Teradata/kylo/pull/90

Boying Lu
August 13, 2018, 2:39 AM

The 'Dat Ingest' based feed doesn't have this issue because the 'feed Id' is cached in the 'Database' processor which will be deleted when deleting the feed.

Done

Assignee

Sean Felten

Reporter

Boying Lu

Labels

None

Reviewer

None

Story point estimate

None

Story Points

2

Components

Sprint

None

Fix versions

Affects versions

Priority

Medium