I have lived with this problem for the last few years. VBA is pretty much dead, yet the VSTO model for me was just like you said: buggy, inflexible, and slow. So I saw it as a choice of getting the job done fast and effectively yet letting my professional skills stagnate, or getting the job done very painfully yet developing skills that will be around in the future.
So tl;dr: if you're primarily an office application developer, stick to vba, learn js when you get time and wait till your company upgrades to Office 2013.
Update August 2015:
JS Office Apps have been out for a few years. During this time I've been waiting for the Api to expand to the same level of functionality as what we have with VBA/VSTO. While I have heard rumors that Office 2016 will bring more api functionality to the Office Apps ecosystem, the api has not really evolved to support your standard LOB automation applications. From what Microsoft has released so far (including their newly revamped Office API's) it appears they aren't focused on this. Speculating here, but it appears Microsoft sees Office Apps really like an interface to the web/cloud/online store. Office Apps are not meant to be an application so much as to provide a window to an application. The focus is also on user interactivity, as opposed to automation. This is fundamentally different than the VBA/VSTO model, which treats Office as the application, it's own self contained application. I don't expect Apps to support the use cases of VBA/VSTO anytime soon. I remember reading Microsoft wants heavy office automation to die out, probably due to the security risks among other things. I use VSTO for Office automation now.