Performance of concurrent software on multicore processors [closed]

Question

Recently I have often read that, since the trend is to build processors with multiple cores, it will be increasingly important to have programming languages that support concurrent and parallel programming in order to better exploit the parallelism offered by these processors (see e.g. this video for some background.).

In this respect, certain programming paradigms or models are considered well-suited for writing robust concurrent software:

• Functional programming languages, e.g. Haskell, Scala, etc.
• The actor model: Erlang, but also available for Scala / Java (Akka), C++ (Theron, Casablanca, ...), and other programming languages.

My questions:

• What is the state of the art regarding the development of concurrent applications (e.g. using multi-threading) using the above languages / models? Is this area still being explored or are there well-established practices already?
• Will it be more complex to program applications with a higher level of concurrency, or is it just a matter of learning new paradigms and practices?
• How does the performance of highly concurrent software compare to the performance of more traditional software when executed on multiple core processors?
• For example, has anyone implemented a desktop application using C++ / Theron, or Java / Akka? Was there a boost in performance on a multiple core processor due to higher parallelism?

EDIT

NOTE that I am not asking for your opinions or for debate but for concrete experiences or information. For example, has anyone written a Scala or Haskell program, compiled it with existing state-of-the-art compilers, and

1. Run it on an Intel Core i3 and measured a certain performance (e.g. 10 seconds on certain input data).
2. Run the same bytecode or binary on an Intel Core i5 and observed a performance boost (e.g. 6 seconds running time) due to the parallel computation of sub-expressions which is possible in functional code?

EDIT 2

SUMMARIZING. Up to now making faster processors meant increasing the clock speed, and no changes in programming paradigm were needed. In the last few years making faster processor has meant adding more cores but this requires we write software differently. My question is whether software developers are starting to switch to new programming paradigms and whether this is bringing the expected performance boost on multiple-core processors.

Answer 1

(This is not a full answer, but it seems too long to fit in a comment.)

There are many factors affecting the adoption rate of "Design for parallelism" in the software industry. Some of them had nothing to do with the benefits. For example, the skill sets and knowledge levels of developers, etc.

One of my observation is that the type of application determines its adoption rate of parallel paradigm. Each software product line (level) or component has one or more "natural domain / paradigm"; that is, the software would be much easier to develop and maintain if it was implemented in a particular paradigm.

If a paradigm switch is necessary in order to parallelize a certain application, chances are that software companies won't find that cost-effective to justify. If that particular paradigm is easily parallelizable, then what you see is that those software would have higher adoption rate for parallel programming.

---

Regarding the list of paradigms, I would like to add Dataflow. All tasks are declared up-front. Each task declares its inputs and outputs before execution. A task is started as soon as all of its input data is available.

Examples of Dataflow paradigm:

• Intel Concurrent Collections (CnC)
• Habanero
• LabView

---

Answer 1:

Having one very successful paradigm is not enough. To bring up adoption rate of parallel programming, parallelism need to be introduced to other paradigms (including the "outdated" ones) as well.

I have seen others who successfully implemented parallelism in a Windows GUI program, by spawning one Windows dialog (each Windows dialog is housed in a thread) per computation task, and exchange data using Windows messages.

---

Answer 2:

This echoes my observation above: if the introduction of parallelism into an application requires that application to be rewritten in an unnatural paradigm, then the complexity of development and maintenance will be increased.

---

Answer 3:

For purely computationally-intensive tasks, the performance gain usually matches very closely with the prediction by Amdahl's Law, provided that all of the computation is done locally on a computer (i.e. not subject to the much slower network I/O traffic.)

That said, you will quickly find out there are non-parallelizable bottlenecks in your applications. Sometimes these bottlenecks are theoretically non-parallelizable, meaning there is no hope of finding a better algorithm.

---

Answer 4:

A personal story. I have written a simple parallelized program that decodes a JPEG file, resize it, and then save it into a custom image file format. Upon testing, I find that the program takes 1.6 seconds to finish, when tested using 3 threads or 4 threads. It turns out the JPEG decoding step is taking more than 25% of the time, making it the slowest non-parallelizable step.

In other words, Amdahl's law takes effect with just 3-4 CPU cores for my little program.

Sometimes these bottlenecks can be removed if you are allowed to change the software requirements (for example, if you could require your customers to not use a particular image format), but most of the time the requirements are set in stone.

Answer 2

You cannot magically make code not written for concurrency work well concurrently (and magically includes using compilers).

Many times conditions have changed and code needed to be done differently. In the very old days, memory was a premium, so self-modifying code was common. These days, the benefits of code in memory being read-only are so great that all modern operating systems want to enforce it.

Also think what happened when Windows became popular. All the DOS programmers had to rethink their programming ways. You could not have a single keyboard polling loop invoking the complete application functionality in a single threaded program - you had to have event handlers - which in turn changed the way the application code was designed.

Mantras like "use immutable objects!" are experiences learned the hard way. The reasons why are frequently lost on the way but can be reconstructed. The "use immutable objects" is a simple way to allow caching data multiple places without having your program break by one of these places being updated.

The change needed to be concurrent does not need to be a full rewrite in a functional language, but can be solved with libraries - which allows to keep existing code libraries - but you still need to write your program to utilize it. The OpenCL (Grand Central in OS X) approach is a very interesting way to use both CPU and GPU's to execute code, but again you need to make your program accordingly.

In Java a lot of effort has been spent on providing good building blocks for transparently scale the executing of small code snippets, but you cannot use it if you do not write for it.