Yes and no. Its helpful to see the word "startup" as meaning two distinct things.
The "current" use of the word implies VC capital. As in "I joined a startup and we raised..."
However in the past it referred to literally "starting a business". Someone sees a gap in the market, builds a product, and looks to fill that gap one customer at a time. Since there is no incoming capital, it has to be revenue positive, well enough for the owners to survive.
This sort of business is (somewhat perjorativly) called a "life style" business. In that it provides salaries to some number of people, and provides a (good?) living.
The reasons for starting such a business are numerous, and while maybe some do with the goal of being mega-rich, in reality the successful ones end up being merely "rich".
But there are a _lot_ of upsides other than money. I'll offer a few from personal experience, but really they offer any goal you like.
A) freedom to work on what you want, when you want, where you want, how you want. Yes, of course you are constrained by actually delivering (or you don't get paid) but the mechanics of delivering are up to you.
B) job security. It takes time to get to this point, but once a business is profitable the owner has a job for life. (he is literally the last person to get laid off) [1]
C) a lack of corporate bullshit. No endless layers of managers. No endless meetings. If you need a new PC you go buy a new PC. (equally true, if you report directly to the owner, decisions get made quickly. If you need a new PC you only have to convince one person.)
D) while on the topic, being employed by a small business has huge benefits as well, but to need to move the income needle for the job security.
I could go on but I think you get the point.
[1] ironically when getting person credit, like a mortgage, ticking "self employed" leads me to a lower credit score, despite the fact that 40-odd people get laid off before I do :)
[2] about 5% of new (bootstrapped) businesses make it to 10 years old. I'm not suggesting this route is easy, but it is desirable.