KYLO-1019

MergeTable ended with broken pipe exception

Description

Data wrangler feed failed at step shown below.
on Kylo 0.8.2 sandbox

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 2017-08-17 12:41:56,396 ERROR [Timer-Driven Process Thread-3] c.t.nifi.v2.ingest.MergeTable java.lang.RuntimeException: Failed to execute query at com.thinkbiganalytics.ingest.TableMergeSyncSupport.doExecuteSQL(TableMergeSyncSupport.java:711) ~[kylo-nifi-core-processors-0.8.2.jar:na] at com.thinkbiganalytics.ingest.TableMergeSyncSupport.enableDynamicPartitions(TableMergeSyncSupport.java:65) ~[kylo-nifi-core-processors-0.8.2.jar:na] at com.thinkbiganalytics.nifi.v2.ingest.MergeTable.onTrigger(MergeTable.java:194) ~[kylo-nifi-core-processors-0.8.2.jar:na] at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.0.0.jar:1.0.0] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1064) [nifi-framework-core-1.0.0.jar:1.0.0] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.0.0.jar:1.0.0] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.0.0.jar:1.0.0] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.0.0.jar:1.0.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131] Caused by: java.sql.SQLException: org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed) at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:262) ~[na:na] at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) ~[na:na] at org.apache.commons.dbcp.DelegatingStatement.execute(DelegatingStatement.java:264) ~[na:na] at com.thinkbiganalytics.ingest.TableMergeSyncSupport.doExecuteSQL(TableMergeSyncSupport.java:708) ~[kylo-nifi-core-processors-0.8.2.jar:na] ... 14 common frames omitted Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe (Write failed) at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147) ~[na:na] at org.apache.thrift.transport.TSaslTransport.flush(TSaslTransport.java:470) ~[na:na] at org.apache.thrift.transport.TSaslClientTransport.flush(TSaslClientTransport.java:37) ~[na:na] at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:65) ~[na:na] at org.apache.hive.service.cli.thrift.TCLIService$Client.send_ExecuteStatement(TCLIService.java:219) ~[na:na] at org.apache.hive.service.cli.thrift.TCLIService$Client.ExecuteStatement(TCLIService.java:211) ~[na:na] at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:253) ~[na:na] ... 17 common frames omitted Caused by: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) ~[na:1.8.0_131] at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) ~[na:1.8.0_131] at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[na:1.8.0_131] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_131] at java.io.BufferedOutputStream.write(BufferedOutputStream.java:126) ~[na:1.8.0_131] at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145) ~[na:na] ... 23 common frames omitted

Looks like same issue as this guy
https://groups.google.com/forum/#!searchin/kylo-community/MergeTable$20$20%22Broken$20Pipe%22%7Csort:relevance/kylo-community/7hQq3YhN1es/JfkmvfcLBgAJ

Status

Assignee

Unassigned

Reporter

Andrew Wu

Labels

Reviewer

None

Components

Priority

Medium
Configure