Scala code to insert a row into Hbase

Question

Below is a case class and a companion object with a method putRecord which adds a new row into Hbase using Put API. This code has some repetitive parts. Is there a way to abstract away the repetitive code

case class HbaseRow(_id: Option[String], field1: Option[Array[String]], field2: Option[Array[String]], field3: Option[String], field4: Option[Long])

object HbaseRow {
    def putRecord(row: HbaseRow): Put = {
        val put = new Put(Bytes.toBytes(row._id.get))

        def addColumn[T](columnName: String,value: T): Put = {
          val columnFamily:Array[Byte] = Bytes.toBytes("cf")
          val col: Array[Byte] = Bytes.toBytes(columnName)
          // Repetative, almost same method call but for different types
          value match {
            case v: String => put.addColumn(columnFamily, col,Bytes.toBytes(v)) 
            case v: Long => put.addColumn(columnFamily, col,Bytes.toBytes(v))
            case v: Boolean => put.addColumn(columnFamily, col,Bytes.toBytes(v))
            case v: Array[String] => put.addColumn(columnFamily, col,Bytes.toBytes(v.mkString(",")))
          }
        }

        //Repetitive will be tedious as columns increase.
        row.field1.map(value => addColumn[Array[String]]("f1",value.mkString(","))) 
        row.field2.map(value => addColumn[Array[String]]("f2",value.mkString(",")))
        row.field3.map(value => addColumn[String]("f3",value))
        row.field4.map(value => addColumn[Long]("f4",value))

        put
    }
}

Mikhail Ionkin · Accepted Answer · 2020-07-26 20:40:00Z

In def addColumn you can extract bytes and then add them to put:

val bytes = value match {
  case v: String => Bytes.toBytes(v)
  case v: Long => Bytes.toBytes(v)
  case v: Boolean => Bytes.toBytes(v)
  case v: Array[String] => Bytes.toBytes(v.mkString(","))
}
put.addColumn(columnFamily, col, bytes)

I'm not sure that Scala can check generic types in pattern matching, so Array[String] may be checked only for class Array. If it is than the compiler will warn you.

2nd, if you don't use result, you can use foreach instead of map.

3rd value.mkString(",") is String so value type is also String, not Array[String]. Of course you can simple declare def addColumn(columnName: String, value: Any), or first create sealed trait ColumnValue and use def addColumn(columnName: String, value: ColumnValue).

According to 3rd, you can use:

List("f1" -> row.field1, ..., "f4" -> row.field4).foreach {
  // you can use only 2nd case if you handle `Array` in `def addColumn`
  case (columnName, field: Array[_]) => addColumn(columnName, field.mkString(","))
  case (columnName, field) => addColumn(columnName, field)
}

Stack Exchange Network

Scala code to insert a row into Hbase

1 Answer 1

You must log in to answer this question.

Hot Network Questions

Scala code to insert a row into Hbase

1 Answer 1

You must log in to answer this question.

Related

Hot Network Questions