I know CGI spawns a new process for every HTTP request that it serves, while servlets use one Java process to serve many requests. What do other technologies (rails, WSGI/django, mod_php, asp.net etc.) do so that web servers do not have to spawn a new process new HTTP request?
Servers generally either
1) Use a thread pool with a thread per client connection request, potentially with a separate pool of separate thread/processes to handle "application" code (as opposed to the basics of handling a HTTP request. A thread handles each request from beginning to end; often thread pooling is used to reduce the amount of startup and teardown required with each connection.
Or, 2) they will use an event-based model which allows a single thread / process to handle many connections. In this case, each of the "events" such as new connection or new received packet will go into a queue of sorts to be worked through by one or a small number of threads. This is a highly efficient way to scale if you can limit the amount of work to be done processing each event (otherwise small quick requests will be queued behind larger requests).
The major difference as far as outcomes are concerned is that both of these models avoid or minimise the overheads of process startup time and inter-process communication and handoffs.
In general, what happens is that servers pool threads to support multiple connections. Once the connection is released, the thread is returned to the pool to be used by the next connection. Now the details of this is different for most things. Thinking about what happens to make this work well, systems will open up a process and reuse it to handle multiple requests once it's returned to the pool.
App Servers bundled inside the web server (ASP.NET, Tomcat) don't have to worry about executing external processes because they built on the assumption that your code will handle threading correctly. When talking about Ruby on Rails for instance, most things are not thread safe. Reuse within in the same process would likely lead to undesirable results. (For rails, at least on JRuby, this likely less of the problem it once was.)