functional programming - The differences between underscore usage in these scala's methods

Question

What is the difference and the term name between these underscore usage from these codes: (see the handler(resource) part)

1.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)_
        hh(2)
    } finally {
        resource.close()
    }
}

val bs = new Array[Byte](4)

readFile(new File("scala.txt")) {
    input => b: Byte => println("Read: " + (input.read(bs) + b))
}

I got compile error:

Error:(55, 29) _ must follow method; cannot follow Byte => T
            val hh = handler(resource)_
                        ^

What does it mean?

2.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource) _
        hh(2)
    } finally {
        resource.close()
    }
}

// Lower parts are same, so removed for brevity...
// ...

The result is same as no. 1, I got: _ must follow method compile error.

I read this is because the underscore is used to convert method to function (ETA Expansion), but I also seen the same underscore is used for Partial Applied Function without problem, for example:

val sum = (x: Int, y: Int) => x + y
val sum2 = sum _

No error in this case.

3.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)(_)
        hh(2)
    } finally {
        resource.close()
    }
}

//...

This one works fine. If I'm not wrong, the underscore in this case is called ETA Expansion, is that correct? But I also read from this Q/A, this kind of underscore is for Partial Applied Function. In same page, someone also said this is a Placeholder syntax. So which one is the correct one?

4.

def readFile[T](f: File)(handler: FileInputStream => Byte => T): T = {
    val resource = new java.io.FileInputStream(f)
    try {
        val hh = handler(resource)
        hh(2)
    } finally {
        resource.close()
    }
}

//...

This one doesn't use underscore but it works fine too just like no. 3. My question for this case, what is the difference with no. 3? Should I use no. 4 than no. 3? Is the underscore redundant in no. 3?

Sorry for a lot of questions here, but that underscore thing is really confusing.

Somehow I thought the complexity of underscore in Scala matches the complexity of pointer and reference (*/&/&&) in C/C++.

UPDATE:

5.

I found something interesting again about the underscore:

scala> def sum(x: Int, y: Int) = x + y     // sum is a method because of def
sum: (x: Int, y: Int)Int

scala> val sum2 = sum _    // sum2 is explicit ETA expanded function from sum
sum2: (Int, Int) => Int = <function2>

scala> sum2(2,3)      // testing
res0: Int = 5

scala> val sum3 = (x: Int, y: Int) => x + y      // sum3 is a function object
sum3: (Int, Int) => Int = <function2>

scala> val sum4 = sum3 _           // what happpened here?
sum4: () => (Int, Int) => Int = <function0>

scala> sum4()(2,3)
res2: Int = 5

Could you tell me what happened to sum4? Why the result of sum3 _ has function type: () => (Int, Int) => Int?

6.

List(1, 2, 3) foreach println _

According to this answer, this is Partially applied functions. Ok, I can see that the space before underscore is kinda tricky. It is actually same as:

List(1, 2, 3).foreach(println(_))

So this is indeed Partially applied function.

But if I did this:

scala> List(1, 2, 3).foreach(println _+1)  //#1
<console>:8: error: type mismatch;
 found   : Int(1)
 required: String
              List(1, 2, 3).foreach(println _+1)
                                          ^

scala> List(1, 2, 3).foreach(println _+"#")    //#2 printed out nothing (why?)

scala> List(1, 2, 3).foreach(println 1+_)      //#3
<console>:1: error: ')' expected but integer literal found.
       List(1, 2, 3).foreach(println 1+_)
                                     ^

scala> List(1, 2, 3).foreach(println "#"+_)    //#4
<console>:1: error: ')' expected but string literal found.
       List(1, 2, 3).foreach(println "#"+_)
                                     ^

Newcomer usually will think the underscore in this case as placeholder, but I believe it isn't, isn't it?

Answer 1

1 & 2 - are same and this is eta-expansion, which means that function is getting converted from just function as part of language to the real object of some FunctionN class:

scala> def f(a: Int) = a
f: (a: Int)Int

scala> f.apply(1)
<console>:9: error: missing arguments for method f;
follow this method with `_' if you want to treat it as a partially applied function
              f.apply(1)
              ^
scala> f _
res1: Int => Int = <function1>    

scala> (f _).apply(1)
res2: Int = 1

It doesn't work in your example as handler(resource) is an expression which returns function-object Byte => T (as handler is a function-object FileInputStream => Byte => T and you did partial applying on it), so scala can't do eta-expansion for expressions (only for values and methods).

4 is partially applied as side effect of scala's curried functions support (by curried i mean ability to take parameters one-by-one).

3 is just explicitly partially applied.

Note that in all 3 examples - your handler: FileInputStream => Byte => T function is an object (so it's already eta-expanded), if you try to do same things with multi-parameter-list methods (which is not yet expanded to the curried function) - you will receive the opposite results for 1&2&4:

scala> def f(a: Int)(b: Int) = a //it's not a curried function, as it's just multi-parameter-list method
f: (a: Int)(b: Int)Int

scala> f(2) 
<console>:9: error: missing arguments for method f;
follow this method with `_' if you want to treat it as a partially applied function
              f(2)
           ^
scala> f(2) _ //you need to convert f(2) to object first
res4: Int => Int = <function1>

scala> f(2)(_)
res5: Int => Int = <function1>

scala> f _  //expand method to the function object
res6: Int => (Int => Int) = <function1>

So partial application also do an eta-expansion for you if needed. You may also think about eta-expansion as (not precisely) function with 0 partially-applied arguments, so it's pretty much close terms, as partially-applied functions are always objects in scala (in haskell it's first-class function) because you always need partially applied function to be first-class-citizen (like object or f-c-function) to apply it after eta-expansion.

5. Scala can do eta-expansion for values itself, as they may be considered as compile-time functions with 0 parameters (that's why you see () => ...). It can expand any value to the function object:

scala> val k = 5
k: Int = 5

scala> val kk = k _
kk: () => Int = <function0>

scala> val kkk = kk _
kkk: () => () => Int = <function0>

scala> 

In your example - value is just another function-object itself. Also (Int, Int) => Int is not fully curried function (it's taking parameters some-count by some-count), but scala can also do automatical partial applying for such. To make it fully curried:

scala> def f(a: Int, b: Int) = a
f: (a: Int, b: Int)Int

scala> (f _).curried
res23: Int => (Int => Int) = <function1>

scala> def f(a: Int, b: Int)(z: Int) = a
f: (a: Int, b: Int)(z: Int)Int

scala> (f _).curried
res22: Int => (Int => (Int => Int)) = <function1>

This process actually called currying.

Another way to make it curried - is using tuples. It's not so pure as currying is actually removing tuples , but scala's Tuple is just a class and not tuple in parameter list: (Int, Int) => Int - input is not a tuple in scala's terminology, but in ((Int, Int)) => Int, input is a tuple (regardless that from FP-perspecive it's a tuple of to objects in first case and tuple of one object in second). Example of pseudo-tuplying:

 scala> def f(a: Int, b: Int) = a
 f: (a: Int, b: Int)Int

 scala> (f _).tupled
 res24: ((Int, Int)) => Int = <function1>

5 vs 1&2 As you've seen before you can't apply eta-expansion to the expression, only methods/values/vars:

 scala> 5 _
 <console>:8: error: _ must follow method; cannot follow Int(5)
          5 _
          ^

 scala> val (a, b) = (5, 5)

 scala> (a + b) _
 <console>:10: error: _ must follow method; cannot follow Int
              (a + b) _
                 ^

You see the "method" requiring in the error message, but scala aims to treat methods/values/vars (when they are members of class/object) in the same way to (at least partially) support UAP.

6 It's eta expansion which returns Function0 by default:

scala> val a = println _
a: () => Unit = <function0>

You may expect function1 here, but println is overloaded and eta-expansion mechanism choose the fewest signature. When other type is expected (like in Function1 in foreach) - it may choose another:

scala> val a: String => Unit = println _
a: String => Unit = <function1>

As i said you may consider function-object as function partially applied with 0 arguments (which includes eta-expansion if needed), so that's the source of confusion with another answer (i would choose better example there).

As i said in P.S.2 this expansion may be applied automatically:

scala> List(1,2) foreach println
1
2

About println _ +"#" - it works because any class (including Function1) in scala has implicit def + (s: String) (that's why Int doesn't work here) defined in Predef (see SI-194):

scala> println _
res50: () => Unit = <function0>

scala> println _ + "#"
res51: String = <function0>#

Every other options doesn't work because of 5 vs 1&2, actually scala can't even parse string after one-parameter function:

scala> println "#"
<console>:1: error: ';' expected but string literal found.
   println "#"
           ^

You should specify object host to fix it as scala expects something like "obj method param" (but this is experimental feature and sometimes you need to paste some empty lines or ";" to make it work):

scala> Predef println "aaa"
aaa

P.S. About C++ reference/pointer. Function has no value as it's compile-time structure, so compiler is just creating a value for it, that process is called eta-expansion (or eta-abstraction for pure functional). This value may be part of pointer (an object with reference to it) or just reference itself - doesn't matter. What is matter is that function moves from compile (method) to the runtime (f-c-function) here, so it "becomes alive".

P.S.2. Sometimes scala do eta-expansion automatically (like here) when partially applied multi-parameter-list method is explicitly passed as parameter.

P.S.N. You may find some additional info about underscore in @Daniel C. Sobral answer about scala punctuation.