My question: what sort of application requires so many concurrent threads of execution?
1) The fact that a language "scales" means there is less chance you'll have to ditch that language when things get more complex down the road. (This is called the "Whole Product" concept.) Many people are ditching Apache for Nginx for this very reason. If you're anywhere close to the "hard limit" imposed by thread overhead, you'll get scared and start thinking about ways to get past it. Web sites can never predict how much traffic they will get, so spending a little time making things scalable is reasonable.
2) One goroutine per request just the start. There are plenty of reasons to use goroutines internally.
- Consider a web app with 100's simultaneous requests, but each request generates 100's of back-end requests. The obvious example is a search engine aggregator. But pretty any app could create goroutines for each "area" on screen, then generate them independently instead of sequentially. For example, every page on Amazon.com is made up of 150+ back-end requests, assembled just for you. You don't notice because they are in parallel, not sequential, and each "area" is it's own web service.
- Consider any app where reliability and latency are paramount. You probably want each incoming request to fire off a few back-end requests, and return whichever data comes back first.
- Consider any "client join" done in your app. Instead of saying "for each element, get data", you can spin off a bunch of goroutines. If you have a bunch of slave DBs to query, you will magically go N time faster. If you don't, it won't be any slower.
hit diminishing returns when the number of threads/processes is much greater than the number of physical cores
Performance isn't the only reason to break up a program into CSP. It can actually make the program easier to understand, and some problems can be solved with a lot less code.
As in the slides linked above, having concurrency in your code is a way to organize the problem. Not having goroutines is like not having a Map/Dictonary/Hash data structure in your language. You can get by without it. But once you have it, you start using it everywhere, and it really simplifies your program.
In the past, this meant "roll your own" multithreaded programming. But this was complex and dangerous -- there still aren't a lot of tools to make sure you're not creating races. And how do you prevent a future maintainer from making a mistake? If you look at big/complex programs, you'll see they expend a LOT of resources in that direction.
Since concurrency isn't a first-class part of most languages, today's programmers have a blind spot for why it would be useful to them. This will only become more apparent as every phone and wristwatch heads towards 1000 cores. Go ships with a built-in race-detector tool.