Benefit of a startup (depending on the stage) --
You get to be involved more broadly, but you will also be working way more. You will learn a lot.
Downside of a startup -- you are putting a lot of personal time in for others benefit. Unless you are a Facebook/twitter/google/etc. (all based on luck) an early engineer at a startup is basically wasting away 25-50k/year (depending on level) he could be getting at a public company.
I've played the odds, out of about six or seven startups I worked at, one managed to pay out for the previous couple of years of work (and I am was back at market salary).
I love startups. I work with some now in my current role, I've consulted with many others over the years, but if your goal is pay for time, they are the wrong place to go.
One more potential battle story -- discussions for non-founding CTO of a seed stage startup. The personal sacrifice at the time to accept would have been about $70k/year in guaranteed money for sub 10% equity in the company and a ramp from 45ish hours to 60+hours/week. On paper from everything you can find via glassdoor/angels list/hackernews, the numbers were right regarding the offer. However, what needed to be built (in terms of the service) consulting for the same type of gig would have been $150+/hr and probably much safer/lucrative -- but wasn't an option.
Despite adoring the lead angel investor, I passed. I have my own projects going on and battle scars that kick in when I go back to thinking about being an early stage in building out other's startups on salary.