For me, the main sign that it's time to leave a company is when I start fantasizing regularly about leaving the company. My gut seems to be wiser than my head on this stuff. Nearly every time that I've moved on, it's turned out to be the right call in hindsight.
And the one time I actually followed my gut and got a new job, I let my boss convince me to stay on (I really loved the company) and then he sold it shortly after to some jackasses that ruined it. And then when I did leave, I left without a good plan and ended up in a family owned company whose infighting left everyone wondering if they'd have a job the next day. Pure hell.
That sucks, it happens, it's not just you.
One place, I had reason to leave. Coincidentally, a non-startup, who I'd previously chosen the current place over, re-approached me, with an even sweeter deal than before (VP, doubling my TC, and a great team). So I went to an exec of the place where I was, saying what the showstopper org problems were, and that I had urgency to make a decision. They convinced me we'd solve the problems, so I decided to stay, out of general policy of loyalty to whatever team I'm currently on. Tech company battle-hardened veterans can guess correctly how that worked out. :)