Streaming UI enhancements

Description

The following would improve user experience for streaming:

  • Remove streaming from running tab, only show in streaming tab

  • Fix the current flows running count accuracy. Is there a way to pass metric directly from prov?

  • Streaming should show running if its actively processing events including in the jobs KPI. For example, stop the feed and it shows running.

  • Add more time intervals (day, week, month) for streaming stats

  • Slow down the stats query rate hit on Postgres? Appears to be 5-10x/sec

  • If failed flows, can we keep and display latest error messages?

  • Able to toggle between IO statistics and flowfile throughput in top chart

Environment

None

Activity

Show:
Scott Reisdorf
August 3, 2017, 1:16 AM

Updates:
1. Running counts are now pulled from NiFi instead of doing the calcuation between Finished/Started
2. Feed Status as Running/Stopped is now based upon the Running counts above instead of the Feed being Enabled/Disabled

Overview Page
1. Ops Dashboard now correctly reflects Streaming Jobs as Running or not in both the Jobs KPI and in the Feed Health Table.
2. Ops Dashboard Feed Health card shows the # of running flows for the streaming feed in the overview page.

Feed Stats Page
1. Running counts and Feed Status are now accurate
2. A table shows any error messages captured when receiving the stats from JMS

TODO:

  • look at queries for performance

  • add more time intervals

Scott Reisdorf
August 24, 2017, 2:49 AM

pushed off 2 items to next sprint (see )
The 2 items moved to next sprint are:

  • Add more time intervals (day, week, month) for streaming stats

  • Slow down the stats query rate hit on Postgres? Appears to be 5-10x/sec

Done

Assignee

Scott Reisdorf

Reporter

MattH

Labels

None

Reviewer

None

Story point estimate

None

Components

Sprint

None

Fix versions

Priority

Medium