When would you need “hundreds of thousands” of threads?

Question

Erlang, Go, and Rust all claim in one way or another that they support concurrent programming with cheap "threads"/coroutines. The Go FAQ states:

It is practical to create hundreds of thousands of goroutines in the same address space.

The Rust Tutorial says:

Because tasks are significantly cheaper to create than traditional threads, Rust can create hundreds of thousands of concurrent tasks on a typical 32-bit system.

Erlang's documentation says:

The default initial heap size of 233 words is quite conservative in order to support Erlang systems with hundreds of thousands or even millions of processes.

My question: what sort of application requires so many concurrent threads of execution? Only the busiest of web servers receive even thousands of simultaneous visitors. Boss-worker/job-dispatching type applications I've written hit diminishing returns when the number of threads/processes is much greater than the number of physical cores. I suppose it might make sense for numerical applications, but in reality most people delegates parallelism to third party libraries written in Fortran/C/C++, not these newer generation languages.

Answer 1

In a language where you're not allowed to modify variables, the simple act of maintaining state requires a separate execution context (which most people would call a thread and Erlang calls a process). Basically, everything is a worker.

Consider this Erlang function, which maintains a counter:

counter(Value) ->
    receive                               % Sit idle until a message is received
        increment -> counter(Value + 1);  % Restart with incremented value
        decrement -> counter(Value - 1);  % Restart with decremented value
        speak     ->
            io:fwrite("~B~n", [Value]),
            counter(Value);               % Restart with unaltered value
        _         -> counter(Value)       % Anything else?  Do nothing.
    end.

In a conventional OO language like C++ or Java, you'd accomplish this by having a class with a private class member, public methods to get or change its state and an instantiated object for each counter. Erlang replaces the notion of the instantiated object with a process, the notion of methods with messages and maintenance of state with tail calls that restart the function with whatever values make up the new state. The hidden benefit in this model -- and most of Erlang's raison d'être -- is that the language automatically serializes access to the counter value through the use of a message queue, making concurrent code very easy to implement with a high degree of safety.

You're probably used to the idea that context switches are expensive, which is still true from the perspective of the host OS. The Erlang runtime is itself a small operating system tuned so switching among its own processes is quick and efficient, all while keeping the number of context switches the OS does down to a minimum. For this reason, having many thousands of processes isn't an issue and is encouraged.

Answer 2

Convenience. Back when I started doing multi-threaded programming, I was doing a lot of simulation and game development on the side for fun. I found it to be of great convenience to just spin off a thread for every single object and let it do it's own thing rather than process each one through a loop. If your code isn't disturbed by non-deterministic behaviour and you don't have collisions, it can make coding easier. With the power available to us now, if I were to get back into that, I can easily imagine spinning off a couple thousand threads due to having enough processing power and memory to handle that many discrete objects!

Answer 3

Some rendering tasks spring to mind here. If you're doing a long chain of ops on every pixel of an image, and if those ops are parallelizable, then even a relatively small 1024x768 image is right into the "hundreds of thousands" bracket.

Answer 4

For Erlang it is common to have one process per connection or other task. So for example a streaming audio server might have 1 process per connected user.

The Erlang VM is optimised to handle thousands or even hundreds of thousands of processes by making context switches very cheap.

Answer 5

one use case - websockets:
as websockets are long-lived compared to simple requests, on a busy server a lot of websockets will accumulate over time. microthreads give you a good conceptual modelling and also an relatively easy implementation.

more in general, cases in which numerous more or less autonomous units are waiting for certain events to occur should be good use-cases.

Answer 6

It might help to think of what Erlang was originally designed to do, which was to manage telecommunications. Activities like routing, switching, sensor collection/aggregation, etc.

Bringing this into the web world - consider a system like Twitter. The system probably wouldn't use microthreads in generating web pages, but it could use them in its collection/caching/distribution of tweets.

This article might be of further help.