I have a Java application that connects to an Apache Spark cluster and performs some operations. I'm trying to connect to a Databricks cluster on Azure, using databricks-connect 7.3. If I run from the terminal databricks-connect test
, everything works perfectly. I'm following their documentation, I included the jars in IntelliJ, added spark.databricks.service.server.enabled true
to the cluster in Databricks and used the following to create the SparkSession:
SparkSession spark = SparkSession
.builder()
.master("local")
.getOrCreate();
The problem is that this command connects to a local cluster that is instantiated at runtime, and does not connect to the remote Databricks cluster. Am I missing something?
question from:
https://stackoverflow.com/questions/66048649/databricks-connect-java-connects-to-local-instead-of-remote 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…