While ingesting data using Data Ingestion template from multiple subdirectories, it seems that the job statistics are not populated correctly:
there is only 1 absolute path per job
there is only 1 filename per job
I tested with 3 files under 3 directories, and it created 2 job statistics for the exact timestamp.
Expected: 1 job with a list of files processed, and a list of subdirectories processed
Attached you can find the stats for job 1, 2 and ingested data.
Source = Filesystem
Input Directory = /var/dropzone/
File Filter = .*
Recurse Subdirectories = true
The last 2 digits of the filename match the station_id first 2 digits, to follow from where the data came from.