Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I have searched for many websites for a definition of "task pool", but most descriptions are very vague. Anywhere I can find a precise definition of it?

More details.

  • Are dependencies allowed in task pool?
  • Can a running task exchange data with other running tasks?
    • To phrase conversely, are tasks permitted to accept data only prior to start and permitted to publish results only after the task has finished?
  • What keeps track of the temporary resources (memory, CPU etc) that are used by tasks?
  • How to define weights that will encourage efficient task assignment?
    • For example, if one task needs to pass a lot of data to the next task, it may make sense to run the next task immediately following the first task, and as close to the first thread / CPU / machine / cluster (locality) as possible
  • What about tasks that are not known (do not exist) in the beginning, but were "created" later?
share|improve this question
    
is that something related to technology or project management? –  Tech Jerk Nov 8 '10 at 6:14
    
@Sri Kumar: it is related to parallel computing. I have some computing task in mind, and I want to know whether it is a task pool or not. However, before I can answer the second question, I need a definition for task pool first. –  rwong Nov 8 '10 at 6:23
add comment

4 Answers

up vote 1 down vote accepted

There's some terminology trouble here, which is why your internet searches haven't yielded the answer yet.

A pool is the name given to a collection of 'expensive', re-usable resources such as threads, completion ports, database connections. The operating system, library or framework will manage the number of items in the pool according to some heuristics based on current load or availability of resource.

Tasks are usually units of execution, such as a Deferred Procedure Call, Delegate Call or Objective-C's NSTask. The simplest case is that there's a queue of these waiting to happen built up by code that needs things done 'later', at lower/higher priority than other tasks, asynchronously, concurrently, on another thread (e.g. UI thread) or some combination of all these. Some frameworks allow for inter-dependencies of tasks to be constructed, so you end up with fairly complex decisions being made to determine which task to execute next, but that's all down to the framework/library/O/S.

As already mentioned, the Task Parallel library is one example of breaking stuff up to be executed. Another is the NSOperationQueue on Mac OS X, which has been leveraged to greatly reduce the number of threads (expensive) required to be running at any one time.

To answer your questions specifically:

  1. Dependencies depend on the framework/library you use, but its' a first-class concept.

  2. Ownership of data is generally de-coupled from the creator of the task and passed to the task itself, as the actual thread/CPU it runs on is unclear. One task handing its' data to another isn't a particular problem conceptually - it depends on the framework for constraints. Use execution dependencies if one task needs the output of another.

  3. As the task owns the data, it should take responsibility for cleaning up afterwards. Clearly, if you're handing data to another task, you need reference counting or garbage collection on top of a data ownership policy.

  4. Some frameworks will allow tuning of which tasks run, how much CPU, at what O/S scheduling level, etc. If a task starts using all available memory, it's likely you'll have to deal with that in your own code. Even if you didn't use a 'task pool' strategy, that'd be a problem.

  5. Tasks created 'later' can be enqueued in the existing task pool for execution.

share|improve this answer
add comment

I'm pretty sure you won't find one authorative answer, because it's a term that could mean different things in different contexts.

In terms of C# 4.0 and the Task Parallel Library, a task pool is a collection of pending work items that need to be run.

To grossly simplify the situation(*), tasks are taken from the pool and run by various worker threads in parallel.

(*) In the actual implementation, Tasks aren't taken from the pool one by one as that introduces too much overhead. Instead, they're taken in batches - and not necessarily in the order they were added to the pool.

share|improve this answer
add comment

A very generic answer is that a task pool is very similar to a priority queue.

Depending on the implementation:

  • You may have an infinite number of subscribers
  • Dependencies can be resolved and sorted according to priority (for instance, a low priority task that depends on the same thing that a high priority task does might get processed before a medium priority task)
  • Tasks can be fed from a static memory pool, or allocate their own, depending
  • Tasks can be preempted if a higher priority task is added and can typically yield on a voluntary basis as well.

A much simpler version of a task pool can be realized by taking 'priority' out of 'priority queue', where you more or less have a FIFO scheduler that keeps track of time and resources.

I think the term is a little too colloquial to define, clearly, outside the context of the platform, language and even task at hand - no pun intended.

share|improve this answer
add comment

In general, I'd say that a task pool should be made in a way that

  1. all tasks can be executed concurrently, given enough resources (with synchronization on shared resources when necessary)
  2. all tasks can be executed serially, in any arbitrary order
  3. any level of concurrence between 1. and 2. is possible
  4. all possible orders and concurrence levels produce valid results (not necessarily the same)

Given that definition,

Are dependencies allowed in task pool?

Too vague. If you mean tasks depending on other tasks, I'd say no. A system with such dependencies is a rather a queue than a pool.

Can a running task exchange data with other running tasks?

It should not. You cannot rely on another task running at the same time.

To phrase conversely, are tasks permitted to accept data only prior to start and permitted to publish results only after the task has finished?

Not necessarily. Unchanged data (i.e. from an immutable database) can be accessed later. Output can be written at any time, as long as no other tasks depends on it.

What keeps track of the temporary resources (memory, CPU etc) that are used by tasks?

The framework? At least, not the tasks.

How to define weights that will encourage efficient task assignment? For example, if one task needs to pass a lot of data to the next task, it may make sense to run the next task immediately following the first task, and as close to the first thread / CPU / machine / cluster (locality) as possible

IMO when such a dependency exists, there should be one task that does both, instead of two tasks

What about tasks that are not known (do not exist) in the beginning, but were "created" later?

Tasks can create new tasks. Those can be executed at any time after creation, possibly concurrently with the task that created them.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.