Question: How competitive are software engineering positions in startup companies?
Fairly competitive. When the successful implementation of your product rests on someone else's shoulders, you tend to have high expectations.
Will I be hard pressed to find someone who will hire me not having "official" software development experience?
If you can demonstrate that you are a skilled programmer and have the right personality, this shouldn't be a huge issue. However, you do need to have the skills they require. Most startups employ compressed SDLCs, if at all. ("Cowboy coding" and "fly by the seat of your pants" are two phrases that come to mind.) The process doesn't need to be pretty, it just needs to be good enough to get the product to market successfully and as quickly as possible/reasonable. One of the typical pains in the post-startup phase is implementing "better" processes.
Does anyone have any tips for me in achieving my goal?
Make sure you are willing, and prepared, to take the risk of joining a startup. Yes, the payouts can be huge, but failures are a dime-a-dozen. That doesn't mean that your time with a failed startup is necessarily a loss. However, if you're not prepared for the likely death of the startup it will be painful, at the very least.
I would suggest you begin by focusing on your strengths as a quant and looking for startups that could use that expertise. (I would actually suggest looking for established companies that could use those skills, as a software developer instead of a financial consultant, and making the switch to startup software dev-shop later.)