You will likely feel at times like you missed out on career opportunities, and as an ambitious person that will always nag at you. But I believe ambition is a bit of a selfish drive to achieve instead of to serve.
I guess I am just trying to say—you will regret it at times. But you will end up having achieved something for them and yourself that is far more important than anything you could have built in your career. I hope you will look back on your post one day and realize what a great choice you made!