PySpark equivalent of adding a constant array to a dataframe as column

I found the below list comprehension to work

>>> arr=["oracle","java"]
>>> mp=[ (lambda x:lit(x))(x) for x in arr ]
>>> df.withColumn("mk",array(mp)).show()
+------+---+----------+----------+--------------+
|  name|age|      role|experience|            mk|
+------+---+----------+----------+--------------+
|  John| 25| Developer|      2.56|[oracle, java]|
| Scott| 30|    Tester|       5.2|[oracle, java]|
|   Jim| 28|       DBA|       3.0|[oracle, java]|
|  Mike| 35|Consultant|      10.0|[oracle, java]|
|Daniel| 26| Developer|       3.2|[oracle, java]|
|  Paul| 29|    Tester|       3.6|[oracle, java]|
| Peter| 30| Developer|       6.5|[oracle, java]|
+------+---+----------+----------+--------------+

>>>

There is difference between ar declare in scala and tag declare in python. ar is array type but tag is List type and lit does not allow List that's why it is giving error.

You need to install numpy to declare array like below

import numpy as np
tag = np.array(("oracle","java"))

Just for reference if you use List in scala, it will also give error

scala> val ar = List("oracle","java")
ar: List[String] = List(oracle, java)

scala> df.withColumn("newcol", lit(ar)).printSchema
java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.$colon$colon List(oracle, java)
  at org.apache.spark.sql.catalyst.expressions.Literal$.apply(literals.scala:78)
  at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$2.apply(literals.scala:164)
  at org.apache.spark.sql.catalyst.expressions.Literal$$anonfun$create$2.apply(literals.scala:164)
  at scala.util.Try.getOrElse(Try.scala:79)
  at org.apache.spark.sql.catalyst.expressions.Literal$.create(literals.scala:163)
  at org.apache.spark.sql.functions$.typedLit(functions.scala:127)
  at org.apache.spark.sql.functions$.lit(functions.scala:110)

Spark 3.4+

F.lit(["oracle", "java"])

Full example:

from pyspark.sql import functions as F

df = spark.range(5)
df = df.withColumn("tags", F.lit(["oracle", "java"]))

df.show()
# +---+--------------+
# | id|          tags|
# +---+--------------+
# |  0|[oracle, java]|
# |  1|[oracle, java]|
# |  2|[oracle, java]|
# |  3|[oracle, java]|
# |  4|[oracle, java]|
# +---+--------------+

In the above answer are not appropriate.as we are taking the array of literals . In general for any application we have list of items in the below format and we cannot append that list directly to pyspark dataframe .we should iterate though each of the list item and then converting to literal and then passing the group of literals to pyspark Array function so we can add this Array as new column to the pyspark dataframe. Hope the below code helps

from pyspark.sql.types import * 
import pyspark.sql.functions as F 
import numpy as np 
arr1 = np.array(["oracle","java"]) 
print(arr1)
df_test = spark.createDataFrame(data=[('finance',10),('marketing',20),('sales',30),('IT',40)],schema=['deptname','deptid'])
df_test = df_test.withColumn("tags",F.array([F.lit(item) for item in arr1]))
df_test.show(truncate=False)