I frequently get asked why I didn't do Computer Science as a degree. Before I even left school to go to University (I'm British) I was already running my first startup. (I was 16 and it was game hosting company. I peaked off at two full racks worth of physical servers (several hundred virtual servers)).
At that time in my life, it wasn't even possible for me to have a computer science degree so this startup clearly wasn't affected.
It was around this time also (when faced with the prospect of wanting to go to University for a degree) that I look around at all of the courses available to me and did some research on what I wanted to do.
I really didn't like the look of a single CS degree. I was only really interested in the web and most courses which had any focus on "web technologies"/"the web in general" were so outdated. I didn't have anyone to ask for advice so I figured I could probably learn everything I needed to by hand. I still wanted a degree so I picked a degree which I knew would be well regarded for my entire life. I don't regret that at all.
The only gaping holes in my knowledge which I wish I could fill in, would be advanced maths, some cryptography theory and general algorithm knowledge.
I wouldn't say that I've suffered one bit by not doing a CS degree. The startup that I'm working on has applied to YCombinator and I've build up everything I've needed to through self taught work.
Clearly, depending on the field you want to go into, my path might not work for you, but I've not suffered.