Recent questions tagged Scala

0 votes

587 views

1 answer

I haven't seen many examples of the scalaz state monad. There is this example but it is hard to understand ... presentation on the state monad. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

638 views

1 answer

scala - Difference between case object and object

Is there any difference between case object and object in scala? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - Akka Kill vs. Stop vs. Poison Pill?

Newbie question of Akka - I'm reading over Akka Essentials, could someone please explain the difference between Akka ... the other? Many thanks! See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

759 views

1 answer

scala - Doing multiple column value look up after joining with lookup dataset

I am using spark-sql-2.4.1v how to do various joins depend on the value of column I need get multiple look ... if any more details are needed. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

855 views

1 answer

scala - Why is join not possible after show operator?

The following code works fine until I add show after agg. Why is show not possible? val tempTableB = tableB. ... is this behaving this way? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

951 views

1 answer

scala - Joining two dataframes without a common column

I have two dataframes which has different types of columns. I need to join those two different dataframe. Please ... raja 8595456552 22 cattoy 2 See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

863 views

1 answer

scala - Underscore in List.filter

Why this is not working: List(true,false).filter(_).size The error says: <console>:8: error: missing parameter type ... m using Scala 2.9.0.1. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

878 views

1 answer

scala - How to remove parentheses around records when saveAsTextFile on RDD[(String, Int)]?

I'm using saveAsTextFile(path) in order to save output as text file in later to import the result to DB. ... ) How to remove the parentheses? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

835 views

1 answer

scala - Spark-Monotonically increasing id not working as expected in dataframe?

I have a dataframe df in Spark which looks something like this: scala> df.show() +--------+------- ... the id column not properly indexed here? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

991 views

1 answer

scala - How to suppress info and success messages in sbt?

When I do sbt run I see some header and footer info which I would like to get rid of: $ sbt run [info] Set ... 2.9.1 SBT version: 0.11.1 See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

811 views

1 answer

scala - Spark ML VectorAssembler returns strange output

I am experiencing a very strange behaviour from VectorAssembler and I was wondering if anyone else has seen this. My ... with my code. Can you? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.4k views

1 answer

scala - scalac compile yields "object apache is not a member of package org"

My code is: import org.apache.spark.SparkContext It can run in interactive mode, but when I use scalac to ... how to configure the path. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

768 views

1 answer

scala - Unable to map on HList

I was trying to solve this problem with shapeless. However I am for some reason unable to map on the HList. I ... Why doesn't the mapping work? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

658 views

1 answer

scala - Can't access Parent's Members while dealing with Macro Annotations

I am kind of blocked with the following (macro annotation) situation. Suppose I have an annotation called @factory ... for the local values. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

776 views

1 answer

scala - Addition of two RDD[mllib.linalg.Vector]'s

I need addition of two matrices that are stored in two files. The content of latest1.txt and latest2.txt has ... .Vector]s in Apache-Spark. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

793 views

1 answer

scala - Setting textinputformat.record.delimiter in spark

In Spark, it is possible to set some hadoop configuration settings like, e.g. System.setProperty("spark.hadoop ... delimited by two newlines. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

883 views

1 answer

scala - Access Array column in Spark

A Spark DataFrame contains a column of type Array[Double]. It throw a ClassCastException exception when I try to get ... and scala 2.10.6 Thanks See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - Calculate Cosine Similarity Spark Dataframe

I am using Spark Scala to calculate cosine similarity between the Dataframe rows. Dataframe format is below root |-- ... 't do it using Scala See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

846 views

1 answer

scala - Spark textFile vs wholeTextFiles

I understand the basic theory of textFile generating partition for each file, while wholeTextFiles generates an RDD ... benefit over textFile? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

827 views

1 answer

scala - Type inference on method return type

Why does Scala fail to infer the return type of the method when there's an explicit return statement used in the ... String]) = println(who) } See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

scala - How to explode an array into multiple columns in Spark

I have a spark dataframe looks like: id DataArray a array(3,2,1) b array(4,2,1) c array(8,6,1) d array(8 ... d 8 2 4 What function should I use? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

917 views

1 answer

scala - How to transpose an RDD in Spark

I have an RDD like this: 1 2 3 4 5 6 7 8 9 It is a matrix. Now I want to transpose the RDD like this: 1 4 7 2 5 8 3 6 9 How can I do this? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

881 views

1 answer

scala - Modify collection inside a Spark RDD foreach

I'm trying to add elements to a map while iterating the elements of an RDD. I'm not getting any ... that is being modified somewhere else? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - How to compute cumulative sum using Spark

I have an rdd of (String,Int) which is sorted by key val data = Array(("c1",6), ("c2",3),("c3",4)) val rdd = ... = sum + x._2 ; (x._1,temp); }} See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

647 views

1 answer

scala parallel collections degree of parallelism

Is there any equivalent in scala parallel collections to LINQ's withDegreeOfParallelism which sets the number of ... of threads running. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

788 views

1 answer

scala - Why does Spark RDD partition has 2GB limit for HDFS?

I have get an error when using mllib RandomForest to train data. As my dataset is huge and the default partition ... for the limit in the spark See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

550 views

1 answer

scala - Using context bounds "negatively" to ensure type class instance is absent from scope

tl;dr: How do I do something like the made up code below: def notFunctor[M[_] : Not[Functor]](m: M[_ ... Thanks for any help or perspective :-) See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

828 views

1 answer

scala - Exploding nested Struct in Spark dataframe

I'm working through a Databricks example. The schema for the dataframe looks like: > parquetDF.printSchema root |-- ... ($"department") { ^ See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags