I have been in this situation most of my career. Here are the pros and cons as I see it:
- You usually dabble in more than just programming; you also act as a systems analyst, architect, and developer.
- More exposure to the actual business, since you really need to know what's going on due to not having an analyst/architect to handle the design work
- You've hit the glass ceiling in almost every instance of this scenario. Unless you're in a rare company that plans to expand it's IT/Software department, you will never be promoted as there's nothing to be promoted to, and you probably will never get to hire people under you.
- Probably overworked, possibly underpaid
- Easy to get frustrated due to not having anyone to bounce ideas off of or brainstorm, you basically have to go with your gut instinct or spend a lot of time on sites like this or IRC or whatever talking to your peers.
Ultimately, I would say it's not a good career choice, unless you have a vested interest in the business, if it's a business you want to move into from IT, or if you are close friends/relative to the people running the company. You will learn a lot about non-software, but your experience is largely discounted because you're always going to have a heap on your plate. In all but the rarest of experiences, you're never going to be able to improve the company if their setup is bad (i.e. low-quality hardware, bad code/database), and you'll never be able to hire people under you and then become a true manager.
When I worked at a small six-person shop two years ago in exactly this role my title was "Director of IT" but I had no reports and was the sysadmin/developer/DBA all-in-one. While I learned a bit about dealing with senior management on tech decisions, I was never given the resources to actually do anything (i.e. I had no reports, no budget, and was not authorized to make any decisions), and as such I would be laughed out of any real manager interview I would go for since I wasn't a real manager but an "IT Guy" with a grandiose title. The stagnation ultimately did more harm than good and I ended up with a fair bit of health-related issues (ulcers) due to the stress of having to manage a dozen or more things on a shoestring budget with zero support or assistance.
Would I do it over again? Only if I knew the company was solidly behind me taking the reins as an IT Manager, and would give me A) The resources I need to do my job, whether that's a budget for software/hardware, the ability to hire people, or at the least hire good contractors and B) The support to make educated decisions for the company; basically being treated like an executive/manager (because if you are the only IT person, you are essentially the CTO/CIO and are expected to confer almost daily with management about the technology direction of the company) and not like an employee.
Situations like this almost never turn out well unless you find a rare case with business owners who care about growing and want to grow their IT Department. In a case like that, you are gold if you want to become a manager. In most cases though you'll just get burned out and have nothing to show for it except a trumped-up title that no true manager will accept as valid.