Package spark-avro library with spark shell client (v1 and v2)

Description

A recent fix to allow parsing of avro files to get schema information requires access to spark-avro library. Package this dependency (there are specific versions for Spark 1 and Spark 2) along with the spark shell client.

Activity

Show:
Mike Merritt-Holmes
April 17, 2017, 10:22 PM

Thanks

Jagrut Sharma
April 15, 2017, 11:27 PM

You can test this with Spark 1 using the standard-ingest template that comes with Kylo and attached userdata1.avro file.

Mike Merritt-Holmes
April 14, 2017, 10:42 AM

When you have tested this fix it would be great if you could share the file and template you tested with.

Done

Assignee

Jagrut Sharma

Reporter

Jagrut Sharma

Labels

None

Reviewer

None

Sprint

None

Fix versions

Priority

Medium