As you said, you are fresh out of University. Didn't they teach you anything about Computer Architecture? And Network Infrastructure? That seems so strange.
As for your question: it depends. If you are going to be coding mindless business applications for all your entire programming career, you probably need only basic understanding of Technical Infrastructure (resources are limited, you know).
If you are planning to develop some serious networking applications (like router kernel for example), you would need in-depth understanding of underlying hardware as well as network protocols and network infrastructure.
Basically, if you really like programming (do it out of passion), you probably want to learn as much as you can about Software Architecture, Technical Infrastructure and Computer Architecture - this could open the doors to some serious development (firms).
Just want to point out, that side projects does not necessary have to be paid, for example you can contribute to some Open Source project. It could be fun and it will teach you invaluable lesson at the same time.