This post from Python creator, Guido Van Rossum, mentions an early attempt to remove the GIL from Python:
This has been tried before, with disappointing results, which is why I'm reluctant to put much effort into it myself. In 1999 Greg Stein (with Mark Hammond?) produced a fork of Python (1.5 I believe) that removed the GIL, replacing it with fine-grained locks on all mutable data structures. He also submitted patches that removed many of the reliances on global mutable data structures, which I accepted. However, after benchmarking, it was shown that even on the platform with the fastest locking primitive (Windows at the time) it slowed down single-threaded execution nearly two-fold, meaning that on two CPUs, you could get just a little more work done without the GIL than on a single CPU with the GIL. This wasn't enough, and Greg's patch disappeared into oblivion. (See Greg's writeup on the performance.)
I can hardly argue with the actual results, but I really wonder why this happened. Presumably, the main reason that removing the GIL from CPython is so difficult is because of the reference counting memory management system. A typical Python program will call
Py_DECREF thousands or millions of times, making it a key contention point if we were to wrap locks around it.
But, I don't understand why adding atomic primitives would slow down a single threaded program. Suppose we just modified CPython so that the refcount variable in each Python object was an atomic primitive. And then we just do an atomic increment (fetch-and-add instruction) when we need to increment the reference count. This would make Python reference counting thread-safe, and shouldn't have any performance penalty on a single-threaded application, because there would be no lock contention.
But alas, many people who are smarter than me have tried and failed, so obviously I'm missing something here. What is wrong with the way I'm looking at this problem?