In the US, any stock held past December 31 of the year in which it was obtained by exerciing an ISO (the most common kind of startup option) generates "income" according to the Alternative Minimum Tax (AMT).
AMT is essentially a completely different taxation system that runs in parallel to regular income tax in America. If the shares you exercise are worth a good amount, you will probably wind up paying AMT, especially if you live in a place with high state and local taxes.
After calculating your tax liability for both systems, you have to pay whichever is larger of regular income tax or AMT. Usually this payment, or part of it, comes due April 15th of the next year. Your company is not involved in your reporting and paying these taxes, but it's a federal crime to either misrepresent your income (which omitting the exercise from your filing would be) or not to pay the appropriate taxes.
If you haven't realized yet, a lot of this is complicated, so do your own due dilligence and/or work with a tax professional.
The AMT tax rate is 26% or 28%. The "income" considered by AMT for exercising and holding is equal to the difference between your strike price and the fair market value (FMV) of the stock on the day you exercised. FMV is based on a 409a valuation for companies that are still private. For public companies it's the market price for the stock on that day.
I'm not sure what you mean by your having "a low strike price (To make the math easy, I'll give a hyptothetical...
You have 100k vested shares with a strike price of $0.10. The company has done well for the past 3 years and now has a FMV of $3.10 per share. To exercise 100k shares, you would need to pay $10k. However, if you're still holding these shares on 1/1/2021, you will have an AMT tax liability due 4/15/2021 of ~28% * ($3.10 - $0.10) * 100k = $84k.
Now it's weird how AMT works, so the above number isn't exactly what you wind up paying, but it's close. In general, the case stated above implies you would owe a sizeable amount in taxes if you exercised and held the stock.
People do this if they're leaving a company and their options will otherwise expire, or if they strongly believe the company will IPO or be acquired soon. Exercising starts a clock on the stock you're holding. If you hold the stock for a year after you exercise, your earnings will then only be taxed at the long term capital gains rate (probably 20%) rather than the ordinary income tax rate (usually higher than 20%). Earnings are the sale price minus the strike price (what you paid to acquire the stock or your "cost basis").
The downside of exercising now is 1) you have to pay cash to exercise, 2) you will generate a tax liability without necessarily having a way to sell your stock, and 3) if the price of the stock falls, the AMT you paid winds up being an even greater percent of the value. There is a way to receive credit on future tax bills for previously paid AMT tax that wound up like this, but it's complicated and best to be avoided.
The upside is mostly to obtain the a lower tax rate on the eventual sale. Most employees still with the company who didn't exercise their stock early don't exercise it until they plan to sell it.