We made an open source ESP8266 dev board for makers

In business, the law is how the judge will rule, or how the regulator will regulate, or any of the other actions with practical effect on your ability to develop and sell your product. It's fine to have your own philosophy for how the world should be organized, and to advocate for it where you can; but unless and until you convince a regulator, judge, or lawmaker with that advocacy, your philosophy has no bearing on the actual practice of law.

So when you write:

> If it was a low volume prototype I was selling to evaluators before certifying in detail I'd probably say "no, it is certified, it is test and measurement equipment" and then let the company fold in bankruptcy if taken to court. Eventually it may be found the actions of the FCC were unlawful and I might be compensated.

you have broken from reality. Any lawyer you engaged to defend you would explain that you have a losing argument, and that it would be far cheaper to comply than to pay their fees and then lose. Without a lawyer defending you, your odds would be yet worse. If you won, then you'd be lucky to get even a fraction of your legal fees awarded, let alone the lost profits. In every practical sense, you'd be worse off than if you recognized that while the requirement for certification isn't typically enforced, that's nonetheless the law, and that your best action would therefore be to apologize (with a lawyer's help, to avoid admitting any unnecessary guilt) and become compliant.

I'm not saying any of this is good! Enforcement discretion is basically bad, a transfer of power from legislators to enforcers. I'd prefer a system where something like the § 15.23 exemption applied to low-volume commercial products too, since that's what we de facto have and it works fine. But that's not the law now, and wishing doesn't make it so.

You're quite correct that lawyers give conservative advice because they know the cost of conservative compliance is generally lower than the cost of litigation even if you win. The lawyers that I work with sometimes find my tendency to read the regulations to be charmingly naive, because they prefer to simply ask the regulators--in part they're just a bit lazy, but they're also quite aware of the cost (both in legal fees and in friendliness of relationship) to take any conflicting position. I'm again not saying this is good, but it's the system we have.

Finally, if you did want to argue this, I believe you'd be better off saying it's an "evaluation kit" under 47 CFR § 2.803, not "test equipment":

https://www.law.cornell.edu/cfr/text/47/2.803

Per my original comment, I'm not convinced that's a winning argument for this board; but I believe it would be for typical dev boards (that really are used primarily by people evaluating a part for use in a different system, and not as general-purpose single-board computers), and it's at least not a frivolous argument. Note that compliance with the "evaluation kit" rules requires some magic language in the user manual--if you're selling anything that could plausibly fall under those rules, probably a good idea to include that. Note also that you can let customers evaluate unauthorized prototypes as long as you mark them accordingly and don't collect money for the evaluation, and that you can sign a sales contract (but not complete the sale) pre-authorization as long as it's contingent on authorization.

It took me two days of fighting with esp8266_rtos_sdk and hacking together their broken github repo before I finally got anything to work.

esp-idf worked on the first try.