It sure as hell does mean you're rich. Median household income in the US is about $50k/yr. The average global income is about $10k/yr.
"Rich" means more than having a relatively high income. It means having the means to obtain in relative short order, or having already obtained, relative financial independence. An income of $100,000 doesn't qualify in most places. Such a person would be required to find some income if he loses his current source, and he would almost certainly suffer financially and take many hits to his credit during a period without income. This person isn't "rich" until he's amassed enough savings that he can weather such a period without significant risk to his finances or credit, which will take literally decades in most cases.