I have a query in Spark-Scala

Question

I have tried the following in spark-scala.

Logic:

If Code in Data1 = Code in Data2 the record has to be written to the output file. so based on this condition 4*2; 8 rows are getting written.

Is there any way to optimize the below piece of code to group data?

.join(Data1,
 col("Code") === col("Code"), "inner")
 .selectExpr("Id" ,
            "Date",
            "Code"
             ).as[OutData]

Data1
+---------------+------------
|Id             |Code     
+---------------+------------
|0839           |06869242986
|4395           |06869242986
|3796           |06869242986
|3592           |06869242986
+---------------+------------

Data2
+---------------+------------
|Date           |Code     
+---------------+------------
|202050         |06869242986
|202051         |06869242986
+---------------+------------

OutData
+---------------+-------+------------
|Id             |Date   |Code
+---------------+-------+------------
|0839           |202050 |06869242986
|4395           |202050 |06869242986
|3796           |202050 |06869242986
|3592           |202050 |06869242986
|0839           |202051 |06869242986
|4395           |202051 |06869242986
|3796           |202051 |06869242986
|3592           |202051 |06869242986
+---------------+-------+------------

Answer 1

等待大神答复