I have the following question. I want to pass the value of a variable (string) from scala to python in databricks. I know I can transfer dataframe information between the two languages ??using this command:
%scala scalaDF.registerTempTable("some_table") %python spark.table("some_table")
But I can't transfer a string this way, any ideas?
Create one broadcast variable in scala spark context and get it's value in pyspark context.
2.1m questions
2.1m answers
60 comments
57.0k users