This is a hard question. I considered dropping out multiple times in college, but ended up sticking it out and getting my degree. I'm still not sure if it was the right choice, but it seems to be working out so far. Anyways, I have no answers, but I have some observations.
One of my main fears in college was that I would graduate, take a soul-crushing job, and then lose all desire to start a startup. This hasn't happened. I still want a successful startup just as much as I did in college, or maybe even more. You're not going to stop wanting what you've always wanted just because you have a degree, a job, money, and options.
OTOH, having a job and an income fucks with your perceptions a little. I was lucky (smart?) and didn't change my spending habits; over the past year and a half that I've been working, I banked close to 90% of my take-home pay. Even though I'm financially much more secure than I was coming out of college, it feels much more precarious - I've gotten used to having a certain amount of money coming in each month. Quitting my day job wouldn't be an issue if I didn't have a day job.
Don't underestimate the experience you'll learn on the job. I feel like I've benefited significantly from working in somebody else's startup for a year and a half before trying my own. It also creates a sense of momentum and gets your brain in gear; you'll need all the focus you can have when starting your startup.
Don't overestimate the book learning you'll get in college. Honestly, I don't think a single thing I learned in the classroom has helped me with my startup.
OTOH, a lot of college learning occurs outside the classroom. My impossible quest to complete a computer science major in one semester while the department would not let me declare one turned out to be exactly what startup life is all about. There will be many moments when you have to work like hell for what feels like zero payoff. It's a good experience to really fuck up and then recover from it, because you do that a lot with startups. If you skip the recovery part in college, how do you know you'll be able to do it with your startup?
Also, I met my cofounders in college. Had I left after freshman year, I would never have known them. Had I left later, I probably wouldn't have kept in touch.
Probably the biggest factor in my decision not to leave is that I didn't have a clear idea what I would do instead. That's probably the biggest determinant: if you know exactly what you'll be working on instead (like Michael Dell, Bill Gates, or Mark Zuckerberg) do that instead. If you don't, stay in school, because you'll probably end up doing nothing instead, and nothing isn't a particularly good teacher.