Monthly Archives: October 2015

Scala (18) – Concurrency with Futures in Scala

Even though we all want everything goes parallel and each task is independent with others totally, sometimes it is not simply as we think. Everything has an order, what we can do is to let independent things work at the same time, and let dependent things work by order.

A future gives us a simple way to deal with it. Each future is like an independent tasks and it starts running concurrently. But within a future, each steps in a task run by order.

Here we give two examples. one is only one task, the other is multiple tasks.

One Future

import scala.concurrent.{ Future, Await }
import scala.concurrent.duration._
val response1 = Future { create1(item) }
val returnInfo = Await.result(response1, Duration.Inf)

Multiple Futures

val response1 = Future { create1(item) }
val response2 = Future { create2(item) }
val response3 = Future { create3(item) }

val finalInfoTemp = for { 
  t <- response1
  g <- response2
  r <- response3
} yield (t, g, r)

val (t, g, r) = Await.result(finalInfoTemp, Duration.Inf)
val finalInfo = t ++ g ++ r

Note: Await.result will block the thread with a mandatory timeout duration. Blocking is not a good thing– you should block only if you really have to — but this is useful, in particular for testing purposes. blocking in general is discouraged when working with Futures and concurrency in order to avoid potential deadlocks and improve performance. Instead, use callbacks or combinators to remain in the future domain:

val finalInfoTemp: Future[Int] = for {
  a <- aFuture
  b <- bFuture
} yield a - b
finalInfoTemp.onSuccess {
  case x if x > 100 =>
}

 

XML Parse by Scala

Recently, I need to do some social media collectors to scrawling data from multiple sources, like twitter, google news, rss, etc. Twitter already has nice java library to get data, named twitter4j. But for other sources, we need to parse the data by myself. Most of them can return xml format back, like google news and rss. So the rest thing is to know how to parse xml, that’s enough. (Note: even though most files use xml, but its detail structure is different sometimes. For example, some use “channel” -> “item” structure, but others are using “entry” structure.)

  1. Load data, here sourceURL is the xml/rss link. For example, RSS/XML EXAMPLE

    1. import scala.xml.XML
      val xml = XML.load(sourceURL)
  2. Extract each tag

    1. val forecast = xml \ "channel" \ "item" \ "forecast"
      // if you don't know its full path, you can use this one
      val forecast = xml \\ "forecast"
  3. Extract each attribute under one tag

    1. val url = xml \ "channel" \ "item" \ "forecast" \ "@url"
      // if you know url attribute is unique, you can use this one to shorten
      val url = xml \\ "@url"
      // convert NodeSeq to String
      val urlString = (forecast \ "@url").text
      // obtain Node label
      val forecaseLabel = forecast.label

Read More:

http://alvinalexander.com/scala/xml-parsing-xpath-extract-xml-tag-attributes

http://alvinalexander.com/scala/how-to-extract-data-from-xml-nodes-in-scala