Empty parameter functions passing as arguments

Question

Here are two functions to be used as UDFs:

def nextString(): String = Random.nextString(10)
def plusOne(a: Int): Int = a + 1

def udfString = udf(nextString)
def udfInt = udf(plusOne)

If I try to use withColumn, myUDF1 will work perfectly fine with udfInt, but throws: can't use Char in Schema for udfString

Probably cause it uses (Int) => (Int) for udfInt type, which is what udf expects

But treats nextString as type String, which obviously leads to an assumption, that I am trying to extract Chars when I apply the function.

It will work if I do something like:

def myUDF: () => String = udf(() => nextString)

Which seems ugly for something that simple. Is there any way to pass udfString as a function, not as String?

I would just do def udfString = udf(() => Random.nextString(10)) — Raphael Roth
– Raphael Roth, Commented Apr 14, 2018 at 6:49

lev · Accepted Answer · 2018-04-14 10:08:59Z

3

when you write the following code:

def udfString = udf(nextString)

it's the same as writing

val s = nextString
def udfString = udf(s)

this compiles because a string is also a function of Int => Char (see here)

you can tell the compiler that you are passing a function to the udf by:

def udfString = udf(nextString _)

answered Apr 14, 2018 at 6:09

lev

4,1274 gold badges37 silver badges50 bronze badges

Sign up to request clarification or add additional context in comments.

1 Answer 1