While I am not a programmer, I work with a team of them every day. I do UI design, and I have learned a lot while working with them.
What they do is terrifying and intimidating, but also very rewarding if you can find the right kind of company at which to work. If you love what you do, and are passionate about it, you will succeed. There are always going to be difficult times regardless of the career you choose, but the best way to get through them is to make sure you're doing what you love.
You'll likely be scared for a while once you start working (post-college), and that won't go away unless you put in hard work and time and/or get a great job with awesome teammates who will teach you.
The biggest thing to keep in mind is that college will teach you the conceptual stuff. You'll graduate thinking you know how to fix any problem. Make sure you leave room for learning from co-workers.
I recommend starting internships as soon as possible, and volunteer to help on projects whenever possible to get as much exposure to different technologies and methods of problem-solving as possible.
Remember - if you love the work, you'll do just fine.