Actually we do have very little time, its just that the upswing and downswing have different feels to it. You might have noticed this even when you are driving for a vacation. Drive to the picnic spot feels like an eternity, drive back home feels quick.
There are ~52 weeks a year. Which makes a week 2% of a year. That's already a fast enough tick. Weeks do go by fast. An year is like 10% of a decade. Once you are past the age of 40. You indeed have little time left.
Human life is short if you are doing remotely well in life. It can be a suffering if you are not doing great.