Earlier quoted context omitted.
It's impossible for growth to be exponential indefinitely. Eventually it'll end up as a sigmoid curve. Secondly, progress isn't smooth. It comes in phases and leaps, and it's possible for it to stagnate for a long time.
I think we are only limited by the amount of matter and energy in the Universe. We are far from reaching any limit.
5% growth per year and two thousand years later you coloinize thousands of observable universes. The observable universe has a length of 90 billion lightyears.
And no, time dilation won't save you unless you think of dumb ideas like cloning yourself so that the humanity that spreads is considered "you". Otherwise you will only get to go to a single planet.