If you just want to rough it, you can lease Crown Land, fees are basically nothing if you're willing to do the paperwork, most leases last from 10-99 years, some of these leases might be open to foreigners (not sure, might need a legitimate business).
Even better start of an amatuer prospecting company, you could probably lease an island the size of manhattan, again for a very low fee. You won't own the land but 99 years is usually enough.
Or you could just drive really far into the bush, no one will notice.