Hot answers tagged

73

Multi-threading is simple. Coding an application for multi-threading is very, very easy. There's a simple trick, and this is to use a well-designed message queue (do not roll your own) to pass data among threads. The hard part is trying to have multiple threads magically update a shared object in some way. That's when it gets error-prone because folks ...


51

The main reason is that referential transparency (and even more so laziness) abstracts over the execution order. This makes it trivial to parallelize evaluation. For example, if both a, b, and || are referentially transparent, then it doesn't matter if in a || b a gets evaluated first, b gets evaluated first, or b doesn't get evaluated at all (because a ...


30

Concurrency and parallelism are two related but distinct concepts. Concurrency means, essentially, that task A and task B both need to happen independently of each other, and A starts running, and then B starts before A is finished. There are various different ways of accomplishing concurrency. One of them is parallelism--having multiple CPUs working on ...


29

Please consider, that Harper's needs for teaching an introductory CS curriculum class are very different from the needs of a real life project. His job is to teach fundamental concepts (e.g. modularity, parallelism, induction) to freshmen. As such it is very important, that the language (and paradigm) choosen can express these concepts with as little ...


28

The operating system provides certain primitives for this kind of interprocess communication that don't require polling. If process A is waiting on mutex M, the OS knows A can't be run and puts it aside in a bucket of processes waiting for something to happen. When the process holding M releases it, the OS looks at the list of processes waiting for it. ...


23

Multi-threaded programming is probably the most difficult solution to concurrency. It basically is quite a low level abstraction of what the machine actually does. There's a number of approaches, such as the actor model or (software) transactional memory, that are much easier. Or working with immutable data structures (such as lists and trees). Generally, ...


20

If you can count on any mathematical experience, illustrate how a normal execution flow that is essentially deterministic becomes not just nondeterministic with several threads, but exponentially complex, because you have to make sure every possible interleaving of machine instructions will still do the right thing. A simple example of a lost update or dirty ...


16

This is probably a bold claim to make, but I somehow suspect, this Robert Harper never really wrote actual software in his life. All he seems to concern himself with is ML and statical type systems. As big a contribution as that might possibly be, I don't see how his claims about OOP have relevancy. This article is not controversial. Controversy involves ...


14

I think there's a non technical angle to this question - IMO it's an issue of trust. We commonly get asked to reproduce complex apps like - oh, I don't know - Facebook for example. I have come to the conclusion that if you are having to explain the complexity of a task to the uninitiated/management - then something's rotten in Denmark. Even if other ninja ...


12

You get zealots of every stripe. Object oriented programming is not a silver bullet. It never was. What it is, is a victim of hype. Inevitably, people get sick of the hype and a backlash starts to develop (regardless of the actual utility of the paradigm). Twenty years from now no doubt we'll have some other backlash against functional programming.


11

Because with a few exceptions (Haskell) there is no way that the compiler can unwrap a loop. The problem is that each iteration through the loop can modify global state. So doing it in a different order may cause things to break. In haskell you can count on a function being pure, which is to say it does not read or change global state, so they can be ...


11

According to Wikipedia: Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel"). That is, parallelism always implies concurrency. Also, are multi-threaded programs ...


11

Java has changed focus with time. At first it was designed as a simple powerful language, as a reaction to "powerful complex" C++. Some features that were in C++ were intentionally left out, like operator overloading, templates, enums, that were deemed too complicated or relics of the C era, and OOP being at the peak of its popularity, everything was made an ...


10

I came to it from a supercomputing background (generally for scientific and engineering uses). The two main styles of parallelism are shared memory, where one program runs with multiple threads in the same address space -- a frequent way to implement that is OPENMP -- and message passing, with MPI being the most popular software. If you only care about ...


10

When Java was first designed it was considered appropriate to leave out anonymous functions. I can think of two reasons (but they might be different from the official ones): Java was designed as an object-oriented language without functions, so it was not very natural to have anonymous functions in a language without functions. Or at least, this would have ...


10

The two concepts are related, but different. Concurrency means that two or more calculations happen within the same time frame, and there is usually some sort of dependency between them. Parallelism means that two or more calculations happen simultaneously. Put boldly, concurrency describes a problem (two things need to happen together), while parallelism ...


10

Compare POSIX threads and Grand Central Dispatch, for example. I have code that dispatches to four threads in eight lines of code. With POSIX, that would all be an absolute nightmare. On the other hand, CUDA / OpenCL are not about multithreading at all, but about using massive vector abilities. (They can do multithreading as well, but vectorizing is the ...


9

There is a distinction between SIMD parallel programming and the more traditional parallel programming model that POSIX uses. SIMD is the model that CUDA, OpenCL, etc. use. There is a single set of instructions that are executed simultaneously by many threads, each operating on their own pool of data. This is very useful for thing like 3-D graphics, where ...


8

Maybe because even a mid-range Graphics card is a couple of orders of magnitude faster than an 8 core CPU at the tasks it is very good at, which is highly parallel algorithms? 8 Core CPU == 8 threads minus any OS threads that are available for your Application ( and that is only if you actually have 8 REAL cores, Hyper Threading doesn't count! ) 512 ...


8

There are two different but related things here: If your program runs a single thread, fiber sub-process or any other other instruction sequencing mechanism, then it is single-threaded, regardless of whether it runs it always on the same core (affinity) or it cycles through different cores. If, to the contrary, your program runs multiple threads, fibers ...


8

Evidently none of these features are new in the programming world and that made wonder why are getting all these things in Java until now. Because Java has to go through an approval process that involves several high-visibility stakeholders in a process akin to "design by committee", and that process takes time, that's all. Contrast that with other ...


8

What is the "at most once" property? If you're discussing the definition of a term, I think it would be most useful to provide it. As far as I understand it, an atomic action is supposed to guarantee that only two of the possible states are observable: either the action is carried out or it isn't. The operation is not divisible. This is supported by the ...


7

A multi-core CPU is very good at MIMD work (Multiple Instructions Multiple Data). A GPU excels at SIMD work (Single Instruction Multiple Data). That means that if all your threads are running the same function, only on different data, then using a GPU is probably a good choice. A classic SIMD example is image processing, when all threads run the same ...


6

You're missing the point. The concurrent collections aren't there for performance so much. They're there so you don't need to try and do all the locking around a dictionary/queue yourself, since doing so is error prone and tedious. (And they're really there so you don't need to try and implement lockless dictionaries/queues yourself) Using them effectively ...


5

I've been developing concurrent systems for several years now, and I have a pretty good grasp on the subject despite my lack of formal training (i.e. no degree). Many of best programmers I know didn't finish the University. As for me I studied Philosophy. C/C++, C#, Java, etc.). In particular, it can be near impossible to recreate ...


5

I cannot answer this question in full because one can only second guess the vague thoughts of its author. I strongly suspect that this article is about to cause some embarrassment to its author. There is nothing about OOP that would suggest that it is neither modular nor parallel: Modularity: A major facet of OOP is that it is indeed modular (but modularity ...


5

One simple thought experiment to understand deadlocks is the "dining philosopher" problem. One of the examples I tend to use to describe how bad race conditions can be is the Therac 25 situation. "Just slapping a lock on it" is the mentality of someone who hasn't come across difficult bugs with multi-threading. And it is possible that he thinks you are ...


5

Most of the programming languages which we are using now came at the time where single threaded programming and single user interaction is the most used for many applications(ex: stand alone desktop applications). With the raise of web applications, cloud computing and multi user applications now we need more of multi threaded applications. The legacy ...


5

While it's definitely not a silver bullet, .Net 4's Task Parallel Library can simplify some aspects of parallelising your code, once you've already made the decision to do so. To answer your question, there certainly are differences between types of problems, in terms of how much benefit they gain from being parallelised. Some problems are referred to as ...


5

Please define the terms mesh and linear array of processors, and/or provide online references to such definitions. There are different flavors of mergesort algorithms. One remarkable difference is: For merging two sorted halves of size N/2 arrays into a fully sorted N array, does it take exactly one stage (performing exactly N or O(N) comparisons), or ...



Only top voted, non community-wiki answers of a minimum length are eligible