Code can be decently sloppy, but there's likely a strong correlation between good code and good startups. Not because the code made them a good startup, but because the good startups are good at most things.
Many startups will do fine with a rough codebase, and obviously you should value the code accordingly (if it's an API as a service, highly, if it's a physical product with no software component, not so much). But be wary of any startup that's close to so bad you worry it might fail. Good founders will rarely let it tip so far to that side of the scale.
Obviously there are loads of exceptions to this rule. But I think if you want to be a founder of a software driven startup or you want to find a great place to work as a software engineer, aim your expectations higher than feels reasonable and you'll probably land at a decent medium.