Ingest of Parquet files with decimal data type cannot apply standardization/validation rules

Description

The decimal data type doesnt allow for standardizers/validators when ingesting Parquet data.
I suspect the code in "ParserHelper.hiveTypeToDescriptor " is the problem, although the decimal type should be coming in as a "double" hive type when it hits the ParserHelper. More debugging is needed

Its possible the type coming in doesnt match "decmial", but it may add in the format (i.e. decimal(10,2) ) ... more debugging is needed

Activity

Show:
Jagrut Sharma
April 11, 2017, 8:34 PM

decimal was being detected as a complex type and hence UI was not allowing setting of validation/standardization rules.
Fixed the issue.

Done

Assignee

Jagrut Sharma

Reporter

Scott Reisdorf

Labels

None

Reviewer

None

Sprint

None

Fix versions

Priority

Medium