Skip to content

Bug of mapResult in ParIterableLike #104

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dengziming opened this issue May 29, 2020 · 0 comments
Closed

Bug of mapResult in ParIterableLike #104

dengziming opened this issue May 29, 2020 · 0 comments

Comments

@dengziming
Copy link

dengziming commented May 29, 2020

here is my code:

object App {

  val executor: ThreadPoolExecutor = Executors.newFixedThreadPool(1, threadFactory()).asInstanceOf[ThreadPoolExecutor]

  val executionContext: ExecutionContextExecutor = ExecutionContext.fromExecutor(executor)
  
  /**
    * Create a thread factory ,sets the threads to daemon.
    */
  def threadFactory(): ThreadFactory = {

    runnable: Runnable => {
      val backingThreadFactory = Executors.defaultThreadFactory
      val thread = backingThreadFactory.newThread(runnable)
      thread.setDaemon(true)
      thread
    }
  }
  
  def main(args: Array[String]): Unit = {

    val par = Array(1).par
    par.tasksupport = new ExecutionContextTaskSupport(executionContext)
    // par.foreach(println(_))
    par.flatMap(i => Array(1)).foreach(println)
  }
}

here is the screenshot from jconsole

  1. the thread I created in my code submit a Task to a ThreadPool and return a future
    image

  2. the thread in ThreadPool submit a Task again to the ThreadPool, since the ThreadPool size=1, so it will block infinitely.

image

I inspect the code of scala.connection.parallel.ParIterableLike.flatMap and find this procedure is troublesome: new FlatMap mapResult { _.resultWithTaskSupport }.
I think this is creating 2 tasks like a tree, with ResultMapping as leaf Node and FlatMap as root Node, first we submit FlatMap to ThreadPool which depends on ResultMapping, so we submit ResultMapping to the ThreadPool but there are no resources in ThreadPool.

So I don't think it's necessary to add mapResult as a leaf node, the code in which can just be moved out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants