It is not very difficult to separate improvements from the land. Critically, due to the deadweight loss of present taxes, even an inaccurate LVT would be superior to the present tax system. The margin of error for LVT is essentially "is it worse than the present system?" I'd recommend reading -
Can Land be Assessed at Highest and Best Use? - https://www.jstor.org/stable/43817496?read-now=1&refreqid=ex...
and
Administrative Simplicity - https://www.jstor.org/stable/43817496?read-now=1&refreqid=ex...
and https://www.jstor.org/stable/43817496?read-now=1&refreqid=ex...
As a simple practical matter, the "Highest and Best Use" is ultimately subservient to "demand for land". Simply because you -could- build the Empire State Building in the desert does not make it so that every plot of land in this desert is assessed at this level. In fact, if you were to build the Empire State Building in the middle of nowhere, the rental value of the plot and the surrounding plots would remain at or near $0. It is inaccurate to suggest that the improvement gives the land its value. People give the land its value, and the location value is not equal to the improvements. Why would I pay someone else rent (which is ultimately what we're discussing here) when I could make those same improvements elsewhere? Louis F Post gets into this point below
It is also important to recognize that the principle of assessment should be seen more about adjacent activity not your own activity. See: http://www.wealthandwant.com/themes/Neighbors%27_Actions.htm...
"Q6. If a land-owner builds, does not that increase the value of his land and consequently the amount of the tax he would have to pay? If so, would not he be taxed for his improvement?
A. No. Upon the value of the building he would never pay any tax. It is true that his improvement might attract others to the locality in such numbers as to make land there scarcer and consequently dearer. His own lot would in that case rise in value with the other land and be taxed more, just as the rest would be. But that would not take any of his labor in taxes; he would still have his building free of taxation. Thus: If on a lot worth $1000 a building worth $1000 were erected, making the whole worth $2000, the tax would fall only upon the $1000 which represents the value of the lot. If land then became so scarce that the lot rose in value to $1500 the tax would be raised. But the owner's improvement would be still exempt. When his property was worth $2000 he was taxed on $1000, the value of the lot, leaving $1000, the value of the building, free; and now, though he is taxed on $1500, the value of the lot, $1000, the value of the building, is still free."
and
https://masongaffney.org/publications/G44Philosophy_of_Publi...
Land and its value is the joint product of at least three things:
• nature, which created it;
• government, which acquired it from other sovereigns and protects it from other powers and extends public works for the public's benefit;
and
• synergism, which is the increment to value that spills over from social and economic activity in the neighborhood of each parcel of land.
Value stemming from all these elements is regarded as unearned by the individual landowner. It is the product of outside forces and therefore a fit object of taxation