hive - oozie workflow triggering a spark action failing for local scratch dir not found error

Question

I am trying to execute a spark action from an oozie workflow. The spark action reads a Hive table data and loads to a HDFS path. But when running the oozie workflow, I am stuck with the below error:

Launcher exception: java.lang.RuntimeException: java.io.FileNotFoundException: File : /homes/bau-vzw_dev_rld/hivelogs/localscratchdir does not exist; org.apache.spark.sql.AnalysisException: java.lang.RuntimeException: java.io.FileNotFoundException: File : /homes/bau-vzw_dev_rld/hivelogs/localscratchdir does not exist;

The above error comes despite the above path ("/homes/bau-vzw_dev_rld/hivelogs/localscratchdir") exists in local FS.

Please help on this.

Answer 1

等待大神答复