I don't think very many people can answer your query as there are some pretty big assumption hurdles present in it. I would recommend studying Functional Programming in general and Scala in particular to deeply understand the kinds of breakthroughs occurring which are strongly related to what it appears you desire.
To succinctly rephrase your question I think you may be asking:
Please tell me about a personal experience I have had where I worked
on a problem using traditional single-threaded assumptions where I
know that I would experience an improvement with a "magically parallelizing" compiler.
I think the challenge is the assumptions (many) required to jump from an existing assumed single-threaded code base to understanding all the different, complex and hard intricacies which are present to take the same program and accurately parallelize it via a compiler. This is a non-trivial jump, one that almost no-one can realistically make without having done both types of implementation.
Having said all of that, I do think there are some software engineering practices that might help you move closer to what you might be seeking. And what I sense you are seeking is to spend as little time as possible having to redesign your single-thread assumed code into something that can be easily parallelized. And that requires you become aware of several domains of work which will support this.
- You must pick a language which naturally supports the notion of parallel code.
- The language must also have libraries for which easily converting single-threaded code to be able to be multi-threaded is a simple class/method/function renaming.
- You need to ensure the language BY DEFAULT supports immutability, and then strongly practice immutability within all of your single-threaded assumed software design.
If you follow all three of these practices, you will find the complexity of moving between single-threaded assumed code and parallelizing it will be dramatically reduced. Absent any one of the three practices above, you will find the hurdles to moving from single-threaded to multi-threaded to be so complex as to be not worth the orders of magnitude of extra effort required. And it's even less likely a compiler can be written to make said jumps.
And I am speaking from personal experience. I started a Java project in 2000 based on roughly the same reasoning you used above. It was an ANN/GA (Artificial Neural Network/Genetic Algorithm) distributed computing system to attempt to create a Go (game) AI. Go being quite a challenge, I started with Checkers first to ensure my system worked. For every hour of useful "single-threaded" work I did, I spent another 99 hours (no exaggeration) on technical tangents unrelated to the core goal. I eventually got my entire system working across 10 nodes. However, after having spent close to 2000 hours working on it, I was completely burned out on attempting to do the Go implementation.
I've since sat back and remained very interested in how I might be able to redo the 2,000 hours of work and reduce it to something on the order of 20-50 hours. I finally resorted to inventing my own language and libraries to see if I could solve it faster that way. Talk about a huge technical tangent, huh? :)
Right after I started generating my list of desirable requirements in 2010/Dec, a friend of mine asked me why I was doing all that work. And then he suggested I take a look at Clojure and Scala. I quickly read up on Clojure and didn't like how noisy/boilerplate-ish it felt coming from Java. I then read up on Scala. And I couldn't believe that it had over 60% of the kinds of features I wanted myself. I then purchased the newly released "Programming in Scala, 2nd Edition" and completely read the e-reader version before I got the physical copy.
I've spent all the time between then and now working very hard to grok Scala, Functional Programming and thinking exclusively in terms of immutability. It's been quite challenging to my decades of OO experience. However, I think I am finally rounding the corner on a couple of the core challenges...finally!
Will I use Scala (and Akka, Play and Scala-IDE) and completely re-create my AI system so I can continue on to my Go goal? I'm playing around with it now. I still have so much to learn and so much more confidence to gain before I'm able to code as quickly in Scala and it's libraries as I can in Java with its libraries.
Anyway, it seemed you wanted a personal story from which to draw your conclusions about the possible profitability of a "single-threaded assumed code based automagically parallelized" compiler. Hopefully, this helped.