@hackage / pooled-io

Run jobs on a limited number of threads and support data dependencies

Latest0.0.2.5

About

Metadata

  • Last updated , by HenningThielemann
  • License BSD-3-Clause
  • Categories Parallelism

  • Maintained by: Henning Thielemann <haskell@henning-thielemann.de>

  • Lottery factor: 1

Links

Installation

Package Flags

Use the -f option with cabal commands to enable flags

    buildexamples (off by default)

    Build example executables

Readme

The motivation for this package was to run computations on multiple cores that need to write intermediate results to disk. The functions restrict the number of simultaneously running jobs to a user given number or to the number of capabilities the Haskell program was started with, i.e. the number after the RTS option -N.

There are some flavors of this functionality:

  • Control.Concurrent.PooledIO.Independent: run independent actions without results in parallel

  • Control.Concurrent.PooledIO.Final: run independent actions with a final result in parallel

  • Control.Concurrent.PooledIO.InOrder: run jobs in parallel with data dependencies like make -j n

Additionally there is the module Control.Concurrent.PooledIO.Sequence that helps to serialize I/O actions from multiple threads. It is certainly most useful in connection with Control.Concurrent.PooledIO.Independent.

Related packages:

  • lazyio: interleave IO actions in a single thread

  • async: start threads and wait for their results, forward exceptions, but do not throttle concurrency with respect to number of available cores

  • parallel-tasks:

  • parallel-io:

  • threadPool:

  • threads-pool:

  • cio:

  • Control-Engine:

  • taskpool, async-pool: