Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Following along with the principle of not optimizing too early, I'm wondering at what point in the design / development of a piece of software do you start thinking about the concurrency opportunities?

I can well imagine that one strategy would be to write a single threaded application, and through profiling identify sections that are candidates to run in parallel. Another strategy I've seen a little of is to consider the software by groups of tasks and to make the independent tasks parallel.

Part of the reason for asking is that of course, if you wait until the end and only refactor the software to operate concurrently, you may structure things in the worst possible way and have a major task on your hand.

What experiences have helped to determine when you consider parallelization in your design?

share|improve this question
up vote 4 down vote accepted

The thing about thread tasks is you want high cohesion, low coupling, and good encapsulation. Interestingly enough, those happen to be worthy design goals for single-threaded applications as well. I had a task today that I didn't originally plan on parallelizing, but when I did, it involved little more than renaming a function to run() and changing how it was called.

Not optimizing too soon means don't put everything in a thread "just in case," but neither should you paint yourself into an architectural corner so it will be too difficult to optimize should the need arise.

share|improve this answer
Yes, re-entrant functions, work queues and that like can help in single threaded as well as in multithreaded applications, but they also allow for good extensibility toward parallel processing later. Which saves a lot of headache with synchronization problems later. – Coder Aug 25 '11 at 23:41

It varies with the project. Sometimes it's very easy to see what can be made parallel: perhaps your program processes batches of files. Suppose that the processing of each file is completely independent of all the other files so it might be quite obvious that you could process 1 file at a time, or 10, or 100, and none of these jobs will impact the other.

It gets a little more complicated when the potential parallel jobs aren't the same. Processing an image file, you could have one job that creates a histogram, another that produces a thumbnail, and maybe another that extracts EXIF metadata and then a final job that takes the output of all of these jobs and stores them in a database. In this example, it's maybe not clear if these should be run in parallel, or if they should (the last job will have to wait for the prior jobs to all complete successfully).

In my experiences, the easiest way to parallelize something is to look for processes that could be run as independently as possible (like in the first example) and start with those. I'd only try to make the second example run in parallel if I thought I'd make a significant gain in performance with it.

share|improve this answer

You must design concurrency into your application from the beginning. Normally as an optimization I would agree that it should be left until later if not inherently obvious. The problem is that concurrency may well require re-architecting your application from scratch in the worst case- some systems are virtually impossible to have concurrency tacked-on. An easy example of this is systems which share data- for example, the simulation and rendering aspects of a game.

share|improve this answer

Java programmers should embrace the Callable interface for work units. If your whole application consists of a loop of creating Callables, shipping all the units of to an Executor, handle any post generation tasks, you have something that can be very easily be shaped into serial processing, "three work queues" and "do all at once" simply by picking the right executor.

We are slowly adapting this pattern as it is very common that the serial approach gets too slow at one point and then we need to do it anyway.

share|improve this answer

I would say that threading is a part of the architecture of the application. So it is one of the first things I need to think of.

E.g. when I do a GUI application, the GUI code is single threaded so long running tasks (e.g. XML processing) would block the GUI, and should be run in a background thread instead.

E.g. a server would either be thread-based, where each request is handled by a new thread, or the server could be event-driven and use only one thread per cpu-core, but then again, long running tasks should be run in a background thread or be divided into smaller tasks.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.