With a simple search in amazon one can see that the modern approach for parallel programming is to use your graphic card. However I am still a little bit skeptical about it. My last computer has an 8 core CPU which I need is enough for basic all my parallel needs, if I need more I will probably use MPI through a network using my old machines. All in all, Why and/or when should I use CUDA or another method which uses my graphic card instead of traditional methods like pthreads, java threads, boost threads or the new C++ 11 threads? What about using processes?
Maybe because even a mid-range Graphics card is a couple of orders of magnitude faster than an 8 core CPU at the tasks it is very good at, which is highly parallel algorithms?
8 Core CPU == 8 threads minus any OS threads that are available for your Application ( and that is only if you actually have 8
512 Stream GPU = 512 Streams for your application to use.
Also the memory bandwidth on modern graphics cards are at least twice as wide and a few orders of magnitude faster than even the fastest chipset available for a general purpose CPU.
The question is confusing multi-threading with parallel concurrency which are not always used together.
GPUs aren't multi-threaded in the same way a general purpose CPU is.
That is they don't execute multiple instructions per stream, but streams can share data that they work on, which general purpose CPUs can't do well, at least not without compromising in space over time issues. It really is comparing Apples and Oranges.
A multi-core CPU is very good at MIMD work (Multiple Instructions Multiple Data). A GPU excels at SIMD work (Single Instruction Multiple Data). That means that if all your threads are running the same function, only on different data, then using a GPU is probably a good choice. A classic SIMD example is image processing, when all threads run the same algorithm on different parts of the image.