Scala Parallel Processing

Question

I am writing a code which would take a list of database tables from a metadata repo and move data between source and target database as configured in the metadata repository. Below are my requirements requirements:-

Process each of the tables in the list in parallel and move data between source and target.
Generate a report which will be sent via email which will have information on which tables have been moved successfully and which have failed.

Below is a sample implementation of the code. The method futureCalc will have the implementation of moving the data. Below are my questions:-

1.For sending the email report do I need to use Async.result as a blocking session to get the list of reports or is there a non blocking way to achieve this?

2.I have read in multiple forums that Callbacks should be used instead of a blocking API, are callbacks called asynchronously in a separate thread from the main thread?

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Success
import scala.util.Failure
import scala.async.Async._

object test extends App {
  def futureCalc(i:Int):Int = {
    val sleeptime = i*60
    println("This is start of thread:"+i)
    Thread.sleep(sleeptime)
    println("This is after sleep :"+i)
    if (i<4)
      i*i
    else
      throw new RuntimeException("Bad thing")
  }

  val iniList   = List(1,2,3,4)
  val resultList = iniList map (i => Future{futureCalc(i)})

  resultList map  (i => {while(!i.isCompleted){
    i.onComplete {
      case Success(j) => println("No Issues")
      case Failure(ex) => println("Something went wrong ")
    }
  }
  }
    )

}

Epicurist · Accepted Answer · 2018-03-01 14:20:40Z

1

If you say Scala and database, use Lightbend's SLICK. The results are futures which can executed like this:

val q2 = for {
  c <- coffees if c.price < 9.0
  s <- suppliers if s.id === c.supID
} yield (c.name, s.name)`

See the Slick documentation.

answered Mar 1, 2018 at 14:20

Epicurist

92311 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jayadeep Jayaraman Over a year ago

Thanks @Epicurist but I would like to run a set of tasks on a list of tables, therefore I was creating a list of futures and executing the list of futures in parallel and finally collecting the results.

Collectives™ on Stack Overflow

Scala Parallel Processing

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related