Recent questions tagged Scala

0 votes

601 views

1 answer

scala - Should I use val or def when defining a Stream?

In an answer to a StackOverflow question I created a Stream as a val, like so: val s:Stream[Int] = 1 #:: ... docs use val. Which one is right? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

878 views

1 answer

scala - Scaladoc [use case]

Why do some method descriptions in Scaladoc start with [use case]? Example: scala.collection.immutable.StringOps.++ ... replaced in the future? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

624 views

1 answer

scala - Can extractors be customized with parameters in the body of a case statement (or anywhere else that an extractor would be used)?

Basically, I would like to be able to build a custom extractor without having to store it in a variable prior to ... I'd ask around first :D See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

832 views

1 answer

scala - Zip multiple sequences

I am trying to zip multiple sequences to form a long tuple: val ints = List(1,2,3) val chars = ... number of equal-length sequences easily? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

613 views

1 answer

scala - Spark count vs take and length

I'm using com.datastax.spark:spark-cassandra-connector_2.11:2.4.0 when run zeppelin notebooks and don't understand the ... ).length) //return: 4 See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - how to deal with error SPARK-5063 in spark

I get the error message SPARK-5063 in the line of println val d.foreach{x=> for(i<-0 until x.length) ... Array[String]] to Array[String] ? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

554 views

1 answer

scala - ClassTag based pattern matching fails for primitives

I thought the following would be the most concise and correct form to collect elements of a collection which satisfy ... fix the above method. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

656 views

1 answer

scala - Pattern matching a String as Seq[Char]

In Scala it is possible formulate patterns based on the invididual characters of a string by treating it as a ... code snippet will not work. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

722 views

1 answer

scala - Column name with dot spark

I am trying to take columns from a DataFrame and convert it to an RDD[Vector]. The problem is that I have ... with dot in their names.Thanks See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

655 views

1 answer

scala - How to access parameter list of case class in a dotty macro

I am trying to learn meta-programming in dotty. Specifically compile time code generation. I thought learning by ... patterns to do this? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

600 views

1 answer

scala - What is version library spark supported SparkSession

Code Spark with SparkSession. import org.apache.spark.SparkConf import org.apache.spark.SparkContext val conf = ... pom.xml. Thanks. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

782 views

1 answer

scala - Spark and SparkSQL: How to imitate window function?

Description Given a dataframe df id | date --------------- 1 | 2015-09-01 2 | 2015-09-01 ... regular dataframes be supported in Spark? Thanks! See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

628 views

1 answer

scala - Why does using cache on streaming Datasets fail with "AnalysisException: Queries with streaming sources must be executed with writeStream.start()"?

SparkSession .builder .master("local[*]") .config("spark.sql.warehouse.dir", "C:/tmp/spark") .config("spark. ... App.main(App.scala) Any idea? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

520 views

1 answer

scala - Use case and examples for type pattern with type variable

I found out reading the spec that scala supports binding type variables when doing a type pattern match: Map(1 -> " ... ), type a(in value res7) See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

747 views

1 answer

scala - Spark Kryo: Register a custom serializer

I have a class that implements a custom Kryo serializer by implementing the read() and write() methods from com. ... a way to do this? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

631 views

1 answer

scala - Compile tests with SBT and package them to be run later

Im working with SBT and Play! Framework. Currently we have a commit stage in our pipeline where we publish to ... way to run the tests. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

776 views

1 answer

scala - How can I calculate exact median with Apache Spark?

This page contains some statistics functions (mean, stdev, variance, etc.) but it does not contain the median. How can I calculate exact median? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - How to split parquet files into many partitions in Spark?

So I have just 1 parquet file I'm reading with Spark (using the SQL stuff) and I'd like it to be processed ... don't understand what's going on. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

630 views

1 answer

scala - Slick codegen & tables with > 22 columns

I'm new to Slick. I'm creating a test suite for a Java application with Scala, ScalaTest and Slick. I'm ... some help here. Thanks in advance! See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

723 views

1 answer

scala By-name parameter on a anonymous function

I'm struggling to write an anonymous function with by-name parameter. Here is what i tired. val fun = (x: Boolean ... Any ideas how to do this ? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

798 views

1 answer

scala - Spark SQL - IN clause

I would like to add where condition for a column with Multiple values in DataFrame. Its working for single value ... ("completed","inprogress") See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

940 views

1 answer

scala - How to find max value in pair RDD?

I have a spark pair RDD (key, count) as below Array[(String, Int)] = Array((a,1), (b,2), (c,1), ( ... is org.apache.spark.rdd.RDD[(String, Int)] See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

774 views

1 answer

scala - Coalesce reduces parallelism of entire stage (spark)

Sometimes Spark "optimizes" a dataframe plan in an inefficient way. Consider the following example in Spark 2.1 ... parallelism in such cases? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

623 views

1 answer

scala - How to know if an object is an instance of a TypeTag's type?

I have a function which is able to know if an object is an instance of a Manifest's type. I would like to ... it is done in the old function)? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Failed to initialize compiler: object java.lang.Object in compiler mirror not found

On Ubuntu 16.04, I installed scala: $ls ~/Binary/scala-2.11.8 bin doc lib man $grep -A 2 SCALA ~/.bashrc ... = true. How can I resolve it? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

743 views

1 answer

scala - Spark : DB connection per Spark RDD partition and do mapPartition

I want to do a mapPartitions on my spark rdd, val newRd = myRdd.mapPartitions( partition => { val connection = ... can I achieve this? Thanks! See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

567 views

1 answer

scala - How do I create an explicit companion object for a case class which behaves identically to the replaced compiler provided implicit companion object?

I have a case class defined as such: case class StreetSecondary(designator: String, value: Option[String]) I ... the implicit companion object. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

941 views

1 answer

scala - Spark : How to use mapPartition and create/close connection per partition

So, I want to do certain operations on my spark DataFrame, write them to DB and create another DataFrame at the end ... ). What to do? Thanks! See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags