Recent questions tagged Scala

0 votes

791 views

1 answer

scala - Spark UDF called more than once per record when DF has too many columns

I'm using Spark 1.6.1 and encountering a strange behaviour: I'm running an UDF with some heavy computations ... reducing the number of columns? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

913 views

1 answer

scala - Running tests on Intellij: Class not found

I'm evaluating IntelliJ (13.0.2 133.696) and cannot get jUnit tests to run from within the IDE. My project ... latest EAP, still the same issue. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

671 views

1 answer

scala - Creating an HList of all pairs from two HLists

I'm using shapeless in Scala, and I'd like to write a function allPairs that will take two HLists and return ... not an inlined block of code. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

741 views

1 answer

scala - How to turn off parallel execution of tests for multi-project builds?

I have a multi-project build with tests in sub-projects and in a parent project. The build is aggregated ... of tests, between projects? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

875 views

1 answer

scala - Cumulative product in Spark?

I try to implement a Cumululative product in Spark scala but I really don't know how to it. I have the ... really helpful :) Thank a lot See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

764 views

1 answer

scala - How to get column names with all values null?

I don't have any ideas to get column names when it has null value For example, case class A(name: String ... output. Array, String, RDD whatever See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

749 views

1 answer

scala - How to transpose dataframe in Spark 1.5 (no pivot operator available)?

I want to transpose following table using spark scala without Pivot function I am using Spark 1.5.1 and Pivot function ... ("Day").sum("Sales") See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.0k views

1 answer

scala - Lookup in Spark dataframes

I am using Spark 1.6 and I would like to know how to implement in lookup in the dataframes. I have two ... to use JOIN on both the dataframes. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

592 views

1 answer

scala - What are the hidden rules regarding the type inference in resolution of implicit conversions?

TL;DR: I thought that if the following compiles: implicitly[X => Y] than so will this: (??? :X) :Y ... [String, Nothing] cannot be found. What? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

689 views

1 answer

scala - Change value of nested column in DataFrame

I have dataframe with two level nested fields root |-- request: struct (nullable = true) | |-- dummyID: string ( ... can I do it using Scala? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Dynamically create case class

I am using a csv library that takes a case class and turns it into rows for me to read. The syntax is ... that case class to the csv library? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

569 views

1 answer

scala - What is the suggested way to instantiate a js.Object for API wrappers

For the following javascript API wrapper: @JSName("React") object React extends js.Object { def createClass(init: ... ("content")) } } See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

810 views

1 answer

scala - How to use kafka.group.id and checkpoints in spark 3.0 structured streaming to continue to read from Kafka where it left off after restart?

Based on the introduction in Spark 3.0, https://spark.apache.org/docs/latest/structured-streaming-kafka-integration ... cluster itself in case. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

861 views

1 answer

scala - Apache Flink: Count window with timeout

Here is a simple code example to illustrate my question: case class Record( key: String, value: Int ) object ... when the timeout is reached. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

721 views

1 answer

scala - Overriding arithmetic operators on Int via implicit conversions

Say that, for aesthetical reasons, I want to be able to write: 3 / 4 and have / be a method on a class ... to "disable" the / method on Int? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

977 views

1 answer

scala - Generic Numeric division

As a general rule, we can take any value of any number type, and divide it by any non-zero value ... available to sidestep this limitation? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

714 views

1 answer

scala - Specs2: how to test a class with more than one injected dependency?

Play 2.4 app, using dependency injection for service classes. I found that Specs2 chokes when a service class being ... "9.4-1205-jdbc42" ) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

797 views

1 answer

scala - Why does leaving the dot out in foldLeft cause a compilation error?

Can anyone explain why I see this compile error for the following when I omit the dot notation for applying the ... 't just invoke foldLeft(1) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

829 views

1 answer

scala - Making sense of forall and exists output on empty list

scala> val l = List() l: List[Nothing] = List() scala> l.forall(x=>false) res0: Boolean = true ... confused. could you somebody help explain? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

638 views

1 answer

scala - Why "case class" doesn't need "new" to create a new object

In Scala what is the reason that you don't need to use "new" to create a new "case class"? I tried searching for awhile now without answers. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

896 views

1 answer

scala - How can I call another task from my SBT task?

I'm trying to call the runTask inside of my task and considered this would work: name := "hello" version := "1 ... ideas on how I could do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

760 views

1 answer

scala - Escape string to be passed as regex

I would like to create a function that creates regex matching an arbitrary string given at the input. For example, ... t. What's wrong here? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

945 views

1 answer

scala - How to split a List[Either[A, B]]

I want to split a List[Either[A, B]] in two lists. Is there a better way ? def lefts[A, B](eithers : List ... .collect { case Right(r) => r} See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.2k views

1 answer

scala - How to split string with trailing empty strings in result?

I am a bit confused about Scala string split behaviour as it does not work consistently and some list elements are ... ) Am I missing something? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

803 views

1 answer

scala - Why can't a variable be a stable identifier?

The following def mMatch(s: String) = { var target: String = "a" s match { case `target` => println("It ... is a deeper reason I am missing. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

774 views

1 answer

scala - Using 'case' in PairRDDFunctions.reduceByKey()

This is the syntax for method reduceByKey def reduceByKey(func: (V, V) ? V): RDD[(K, V)] In a word ... @ghik here. but not able to understand See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

723 views

1 answer

scala - Not able to declare String type accumulator

I am trying to define an accumulator variable of type String in Scala shell (driver) but I keep getting ... Double type of accumulator. Thanks See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

1.1k views

1 answer

scala - Spark streaming DStream RDD to get file name

Spark streaming textFileStream and fileStream can monitor a directory and process the new files in a Dstream RDD ... that particular interval? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags