Optimize Feed Creation in NiFi

Reduce the REST calls and optimize some things to improve performance when saving feeds in NiFi

Fix NiFi REST Calls that are slow. Change Kylo to use option2 below:

The NiFi REST API, https://nifi.apache.org/docs/nifi-docs/rest-api/index.html, has 2 calls you can use to get data about the flows, process groups, processors, and connections

• Option 1 using the “/process-groups” api to get either the process group (and its children) or connection information
  ·
  · I found these calls were taking a considerable amount of time and blocking the NiFi UI whenever they were being made.
  · In particular the connections call: “/process-groups/{id}/connections” took over 40 seconds every time and blocked the User Interface for minutes. Using a profiler I saw the heap grew to 6+Gb and was never able to go back down from there, no matter how long NiFi stayed open. When Kylo saved feeds it was making this call a few times.

• Option 2 using the “/flows” to get process group, flow, connection information
  ·
  · This call will return everything about the process group and its children (processors, connections etc).
  · This takes 1-2 seconds to return for large process groups.