Hi,
I am using Stardog spark connector (stardog-spark-connector-1.0.0.jar) for running graph analytics algorithms on Azure Databricks. I have followed instructions given on Stardog documentation Setup and Usage | Stardog Documentation Latest
My Azure databricks configuration is : Driver: Standard_DS3_v2, Workers: Standard_DS3_v2, 2-8 workers, 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12)
But my job fails with following error for all algorithms:
21/12/08 07:06:33 INFO SparkContext: Successfully stopped SparkContext
21/12/08 07:06:33 INFO ProgressReporter$: Removed result fetcher for 6455807286084721632_7483483563914453930_job-125519-run-11-action-3497022421842561
21/12/08 07:06:33 WARN ScalaDriverWrapper: Spark is detected to be down after running a command
21/12/08 07:06:33 WARN ScalaDriverWrapper: Fatal exception (spark down) in ReplId-5997a-285b9-755be-0
com.databricks.backend.common.rpc.SparkStoppedException: Spark down:
at com.databricks.backend.daemon.driver.DriverWrapper.getCommandOutputAndError(DriverWrapper.scala:530)
at com.databricks.backend.daemon.driver.DriverWrapper.executeCommand(DriverWrapper.scala:634)
at com.databricks.backend.daemon.driver.DriverWrapper.runInnerLoop(DriverWrapper.scala:427)
at com.databricks.backend.daemon.driver.DriverWrapper.runInner(DriverWrapper.scala:370)
at com.databricks.backend.daemon.driver.DriverWrapper.run(DriverWrapper.scala:221)
at java.lang.Thread.run(Thread.java:748)
Also I noticed that though the job status is shown as failed in Databricks UI, when the job is completed, the analytics graph is created successfully in stardog DB. So it seems the algorithm is getting executed successfully, but the job is failing when the spark connector jar is trying to stop the Spark cluster. Due to this , my pipeline is failing. Is this a bug in spark connector? Is there any way to get around it?
Thanks!