Land-value tax doesn't have huge inefficiencies like some other taxes. But the hard part is assessing "value". How should land value be decided? The book "Radical Markets" has some good insights on how to find the value of every plot of land by using markets.
One of the funner ones I've heard of is that the owner and payer of the property tax performs the valuation of the land: it makes implementation pretty much free and uncontroversial! The only caveat is that whatever value the owner lists as the value is an amount they must accept for the land in a sale (plus some meaningful fee to drive off trolls).
So, sure, you can list the value of the land of your Pac Heights mansion as $1 and pay a commensurate tax on that. If you do, though, someone can purchase it for $1 and start charging you a hefty use fee.