First, the fire handles would override any delay and cut fuel (and other things) immediately.
Second: the window of time where you don't have enough altitude (aka time) to restart is relatively small. So this could easily be a temporary protection.
It is difficult to find exact data on this but restart to significant thrust seems to be in the 30-60s range. If you run the numbers on climb rate and glide time the possible danger zone is relatively small, a few minutes after takeoff at most.
Is this an extremely rare event? Yes. But most other accident causes are also rare, regardless of whether they are pilot error or mechanical.
For example: you might think no pilot would deploy the thrust reversers in flight but system protection errors and/or mechanical failures have conspired to allow it and a bunch of people paid in blood to learn that reverser deployment in flight at altitude was actually unrecoverable - contrary to conventional wisdom at the time. It turned out everyone was flying around with a "kill everyone now" mechanism. In some cases with a much lower margin of safety than previously believed due to the aforementioned "conventional wisdom" that if it happened it wouldn't be a big deal.
Know what else isn't normally a big deal (relatively speaking)? Accidental shutdown of both engines. Because a single engine shutdown is easily recovered and the aircraft can fly on one engine. And dual engine shutdown is easily recovered with a restart if you have enough altitude. But it turns out there's a small window after takeoff where it is fatal.
Somewhat relatedly shutting down the wrong engine in an engine failure scenario is so common they explicitly train crews to slow down and not immediately shut down an engine after failure because rushing just leads to dual engine loss.