Thursday, 15 April 2010

using apache oozie ssh actions to execute spark-submit, why does the spark application is stuck on state accepted -


I am trying to run several spark applications one after another and schedule them using oozie. I used an SSH verb, which should spark-on the spark-application. When the SPARC application is easy to walk with the server, the application starts, however, when I used the oozie ssh verb to run spark-submit, I used to see a new spark application, but the status of the application was " Approved "and never really starts to walk.

My SSH action looks like this:

  & lt; Host & gt; $ {User} @ $ {host} & lt; / Host & gt; & Lt; Order & gt; Send for spark & ​​lt; / Order & gt; & Lt; Arg & gt; - Driver-memory & lt; / Arg & gt; & Lt; Arg & gt; 6G & lt; / Arg & gt; & Lt; Arg & gt; - Deploy mode & lt; / Arg & gt; & Lt; Arg & gt; Cluster & lt; / Arg & gt; & Lt; Arg & gt; - Class & lt; / Arg & gt; & Lt; Arg & gt; Main.Application & lt; / Arg & gt; & Lt; Arg & gt; - Exposure memory & lt; / Arg & gt; & Lt; Arg & gt; 6G & lt; / Arg & gt; & Lt; Arg & gt; - Master & lt; / Arg & gt; & Lt; Arg & gt; Yarn-cluster & lt; / Arg & gt; & Lt; Arg & gt; Duplicate.jar & lt; / Arg & gt; ... [Jar's logic] & lt; Capture-output / & gt;  

Am I using oozie ssh verbally?

Has anyone tried to try?

Does anyone have any idea why the application never starts?


No comments:

Post a Comment