Take the 2-minute tour ×
Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. It's 100% free, no registration required.

I believe a lot of Fortran is used in HPC, but not sure if that's only for legacy reasons.

Features of modern programming languages like garbage collection or run-time polymorphism are not suitable for HPC since speed matters so not sure where C# or Java or C++ come in.

Any thoughts?

share|improve this question

closed as not constructive by gnat, Martijn Pieters, MichaelT, GlenH7, BЈовић Mar 15 '13 at 14:33

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

8  
C++ doesn't have a garbage collector and it doesn't require you to use runtime polymorphism. –  Jason Baker Dec 5 '10 at 7:14
    
@Jason My intent is to figure out what features of C++ make it a compelling case for HPC. –  Fanatic23 Dec 5 '10 at 7:31
    
@Fanatic23 - I understand. Just wanted to make a note of that. :-) –  Jason Baker Dec 5 '10 at 7:37
1  
@Fanatic Wish I could say yes, but I don't have too much... I do have a bunch of links regarding some performance issues in .NET/functional languages, though. You might be able to piece the concepts together mentally to get a grasp on certain performance limitations: msdn.microsoft.com/en-us/library/0xy59wtx.aspx stackoverflow.com/questions/2909282/… msdn.microsoft.com/en-us/magazine/cc163329.aspx en.wikipedia.org/wiki/Just-in-time_compilation –  Rei Miyasaka Dec 28 '10 at 14:44
1  
I think though, if you need really good response time, what you're looking for is a real-time OS like QNX: en.wikipedia.org/wiki/QNX –  Rei Miyasaka Dec 28 '10 at 14:46
show 8 more comments

9 Answers

up vote 4 down vote accepted

I have seen a lot of Java used for HPC in areas where (1) there is little legacy code, and (2) development time and code quality matter. Typical application domains are finance, data mining or bio-informatics.

It really depends on the application (there is life outside linear algebra), but the performance of recent JVMs is often on par with C code. Sometimes faster when the JVM is able to perform at runtime clever optimisations that static compilers (C, Fortran) cannot do. And definitely faster when there's a lot of symbolic computation.

Given a fixed amount of time for program development, the resulting Java code is consistently faster than C code. HPC in Java definitely makes sense when code is developed or modified frequently. Another important feature is code mobility over different hardware.

You'll find references in http://ateji.blogspot.com/2010/09/java-for-high-performance-computing.html

Regarding the Fortran assumption that two address are unique, we're working on a static analysis tool that will enable similar optimizations for code in high-level languages, but without the "Bad Things May Happen" bit. Contact me if interested.

share|improve this answer
13  
Nitpick: JIT optimizations are available to static compilers if you're willing to do a little work. Both GCC and MS Visual Studio support Profile Guided Optimizations which optimize using saved runtime data. It's a little misleading to suggest there are optimizations "that static compilers (...) cannot do". –  Corbin March Jan 13 '11 at 16:58
add comment

In my years of experience, up to 5 years ago, it has always been Fortran and C. Which one depended mostly on whether the people came more from engineering or more from CS school of thought (I don't know how to put this better, okey? :-)

In what we were doing Fortran was almost exclusively used.

From what I read around nowadays, with the new updates to the Standard F2003/08 and with the introduction of Co-Arrays, it seems to be gaining momentum again.

Also, one, if not somewhat biased article - The Ideal HPC Programming Language

share|improve this answer
1  
+1: your linked article is excellent. –  rwong Dec 4 '10 at 15:41
add comment

I think for real pedal to the metal, the only real choice is Fortran. The reasoning is that the most important thing for the exploitation of low level ILP (Instruction Level Parallism) is memory address disambiguation. The defacto rules in Fortran allow the compiler to determine that two address are unique (and hence the order of loads and store, or even stores and stores can be interchanged without risk of generating incorrect code). C leaves too much scope for overlapping pointers for the compiler to extract as much low level parallelism from the code.

Also, array alignment, w.r.t cache lines, and SSE/AVX boundaries is important to the generation and execution of efficient loops. If arrays are passed via common blocks, the compiler/loader can assure that all arrays start on the same address alignment boundaries, and more efficient SSE/AVX loads and stores can be utilized. The newer hardware can handle unaligned memory accesses, but because the memory access isn't aligned properly partial use of cache lines results in lower performance. Even if a C programmer properly aligns all his arrays, is there a mechanism to communicate this to the compiler?

To summarize, the two most important issues, are the independence of memory addresses, and the recognition by the compiler that the accessed data structures have the same "natural" alignment that the hardware wants. So far Fortran does the best job on those two tasks.

share|improve this answer
2  
I recently did a little experiment, find the pop count of a string of 64000 bits, represented as a unsigned long long array. I used the exact same algorithm using a lot of interesting boolean and packed arithmetic stuff. In C with -O3 it took 10clocks per long long, whereas with fortran Intel Fortran 10.1, with default optimization it was 6.5! And every programmer think C is superior for bit twiddling! Fortran defacto assumptions allow more efficient low level instruction coding to be safely generated. –  Omega Centauri Dec 27 '10 at 18:11
2  
That should read "The defacto rules in Fortran allow the compiler to ASSUME that two address are unique...". The manuals all tell you that the compiler is allowed to assume this, and warn you IN DETAIL that Bad Things May Happen if you violate that assumption. –  John R. Strohm Dec 27 '10 at 18:24
    
Thanks John, that was well stated. –  Omega Centauri Dec 27 '10 at 23:10
add comment

Just some anecdotal note. I haven't done any high performance computing myself.

For calculations (number crunching), Fortran and C. Yes it is for legacy reasons:

  • Ample availability of public domain source code and recipes.
  • Both support MPI.
  • Both languages are compiled.
  • Compilers for both languages are provided by all HPC OSes and vendors.
  • Vectorizing compilers are available.
  • Both requires crazy level of tweaking to get high performance when ported to a different cluster (different memory size, number of CPUs etc)
    • This actually explains why the open source code is important: tweaking is necessary, therefore the original recipe must be written in a language that is good for manual tweaking.

The current trend for number crunching is to write program generators that automate the source code tweaking in order to optimize performance given the cluster characteristics. These generators often output in C.

A second trend is to write in some specialized dialect of C for specific GPUs or Cell BE.

For non-numerical work, such as programs that process data from a database (but not the database itself), it is much cheaper to run on clusters of "commodity" machines without the expensive customized networking equipments. This is usually called "High Throughput Computing". And Python is the #1 language here (using the famous Map Reduce). Prior to Python, batch processing projects can be written in any language, and are usually dispatched by Condor.

share|improve this answer
1  
Could you elaborate a bit on the "crazy level of tweaking" part? –  Rook Dec 4 '10 at 15:32
    
The computing center hires graduate students to rearrange the MPI calls to make it run faster. –  rwong Dec 4 '10 at 15:36
    
(?) First word here, but I guess practices differ. –  Rook Dec 4 '10 at 15:37
    
It was a climate modeling research center. –  rwong Dec 4 '10 at 15:47
add comment

Fortran is most common, primarily due to legacy (people still run old code) and familiarity (most people who do HPC are not familiar with other kinds of languages).

Features of modern programming languages like garbage collection or run-time polymorphism are not suitable for HPC since speed matters so not sure where C# or Java or C++ come in.

That is not true in general. Classical HPC was mostly doing linear algebra with machine-precision numbers. However, modern HPC is increasingly using supercomputers for a wider variety of crunching, like symbolic calculations with arbitrary mathematical expressions instead of machine precision numbers. This places quite different characteristics on the tools you use and it is not uncommon to use programming languages other than Fortran because symbolic computation can be prohibitively difficult without GC and other kinds of optimizing compiler such as OCaml's optimizing pattern match compiler.

For example, read this paper by Fischbacher et al. which says "the authors have strong reason to believe that this may well be the largest symbolic calculation performed so far".

share|improve this answer
    
Fortran is common because many people use supercomputing time to run simulations of physical systems, such as global weather forecasting, and the implementation of the required algorithms in Fortran very clear and concise. –  Sharpie Jan 13 '11 at 16:33
add comment

Fortran, for some good and some not-so-good reasons. For heavy math crunching, a good reason is there are extensive libraries (BLAS, LAPACK) of tried-and-true subroutines, all written in Fortran (though those can be called from C and C++).

A not-so-good reason is the supposed performance advantage of Fortran over C/C++. Optimizers are pretty good, and few people understand that the benefit of optimizing a piece of code is proportional to the percent of time it is busy, which in almost all code is almost zero.

Another not-so-good reason is a culture gap between CS and non-CS programmers. Scientific programmers tend to be taught bad habits in Fortran, and to look down on the CS programmers and the bad habits they have been taught, and who look down on the former.

share|improve this answer
    
"culture gap between CS and non-CS programmers. Scientific programmers tend to be taught bad habits in Fortran, and to look down on the CS programmers and the bad habits they have been taught, and who look down on the former." Partly this is just that they are concentrating on different aspects of the problem. Fortran means FORmula TRANslation, and it is pretty efficient at translating math formulas into code. For the sorts of programming CS types usually do, other languages are superior. –  Omega Centauri Dec 27 '10 at 23:13
1  
@Omega: You're right. The Fortran-taught folks tend to have no concept of formatting, loathe "implicit none", and cram the code together because they still deal with 72-character lines and think making understandable code is for wimps. The CS-taught people create monster pyramids of classes laced with polymorphisms, notifications, and abstractions, when something simple would do the job. So they deserve each other :) –  Mike Dunlavey Dec 28 '10 at 2:11
5  
the quote used to be "the physicists are solving tomorrows problems on yesterdays hardware - while the CS guys are solving yesterdays problems on tomorrows hardware" –  Martin Beckett Jun 16 '11 at 16:11
    
@Martin: I think maybe I heard that somewhere. It sure rings true. –  Mike Dunlavey Jun 16 '11 at 23:26
    
Martin: So, the hardware guys are the most efficient :) –  Dhaivat Pandya Aug 6 '11 at 9:10
add comment

Basically, all the programs which do the actual work of number crunching are still FORTRAN (the old blas, lapack, arnoldi etc are still the one used)... However, when it comes to higher level structure... people are increasingly using C++.

The complexity of the simulation involve huge code and to get any kind of benefit out of writing one is to make it reusable. Also, the concepts used have also become very complex. It almost madness to represent that information using FORTRAN. That's where C++ comes in since it inherently supports Object Oriented Design. However, Run-Time Polymorphism is rarely preferred. People instead almost always use Static Polymorphism (which is implement in C++ with template meta-programming)

Also, now the compilers are really good, hence a lot of optimization is left to the compilers.

share|improve this answer
add comment

I've been working on some VERY calculation-intensive code in (gasp!) C#.

I'm building a GPGPU implementation of FDTD for optical modeling. On a small (128 processor) cluster, many of our simulations take weeks to run. The GPU implementations, however, tend to run about 50x faster - and that's on a consumer-grade NVidia card. We now have a server with two GTX295 dual-processor cards (several hundred cores), and are getting some Teslas soon.

How does this pertain to your language? In the same way that the C++ FDTD code we were using before was CPU-bound, these are GPU-bound, so the (very small) horsepower difference of managed vs native code doesn't ever come into play. The C# app acts as a conductor - loading OpenCL kernels, passing data to and from the GPUs, providing the user interface, reporting, etc. - all tasks that are a pain in the ass in C++.

In years past, the performance difference between managed and unmanaged code was significant enough that it was sometimes worth putting up with C++'s terrible object model to get the extra few percent of speed. These days, the development cost of C++ vs C# far outweighs the benefits for most applications.

Also, most of your performance difference isn't going to come from your choice of language, but from the skill of your developer. A few weeks ago, I moved a single division operation from the inside of a triple-nested (3D array traversal) loop, which reduced execution time for a given computational domain by 15%. That's a result of processor architecture: division is slow, which is one of those faces that you just need have picked up somewhere.

share|improve this answer
1  
c++ has an object model? But it sounds like you should have gone with a script language to write your controllers in - if C# is better than C++ because of dev speed, then python (or lua, etc) is similarly better than C#. –  gbjbaanb Jul 11 '11 at 13:23
2  
@gbjbaanb Not necessarily. This implementation is GPU-bound, but moving to a scripting language could very easily change that. C# is compiled and has a very nice optimizer. Compiled, strongly-typed languages are your friends! Less strict scripting languages tend to cause increased development time for any reasonably complex project. –  David Lively Jul 11 '11 at 15:45
add comment

There are two kinds of problems that need to be addressed in HPC applications: one is the number crunching itself and the other is managing of computations. The first one is usually approached with code written in Fortran, C or C++ because of speed and because of the fact that there are already a lot of scientific algorithms written in this languages. The steering of computations is more conveniently implemented in higher level languages. Python is a "glue" language of choice for handling application logic and calling extensions implemented in compiled languages. Java is frequently used by projects in which managing networking and distributed computing is essential.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.