Earlier quoted context omitted.
Why did you buy a lot zoned ag when you clearly wanted one zoned residential? A little due diligence before purchasing the lot and you would either know what you in for and still buy or decide it's too much of a hassle and walk.
They probably bought a lot smaller than 40 acres and assumed that logically, a single dwelling would be allowed since otherwise a dwelling would be impossible.
If "a dwelling would be impossible", the logical conclusion is that zero dwellings would be allowed.
This situation is the norm in US unincorporated zoning. I have never heard of any unincorporated jurisdiction with a "minimum residences per parcel regardless of parcel size" exception. If there is such a jurisdiction, it's the exception not the rule.
There is a limit to how many dwellings an area can support without major infrastructure upgrades (sewerage, water lines, new roads, additional sheriff's deputies). This limit has to be divided among the parcels. The fairest way to do it is by acreage. "Minimum acres per dwelling" is just the reciprocal expression of "maximum dwellings per acre" and more well-behaved since fractional acres make sense but fractional dwellings do not.
Giving tiny postage-stamp parcels the right to build one dwelling would be a windfall for all the kooky "sliver parcels" created by things like railroads and surveyors' errors. Those obnoxious error parcels are made undevelopable in order to encourage that they be merged into a neighboring parcel.