With guided decoding (structured output according to a schema), a model can sometimes return broken JSON - usually if it stops midway for some reason. In those rare cases, a better approach would be to simply retry, no? Trying to "fix" broken JSON without understanding the context can mask real problems and produce data that appears to be correct, but is actually corrupted.
Response Healing: Reduce JSON defects by 80%+
51–52 of 52 posts
Re: Response Healing: Reduce JSON defects by 80%+
#52Is this a joke? Am I going crazy? I don't like this future we're going towards where we have to trick our software (which we can no longer understand the workings of) into doing what we tell it to by asking it nicely, or by putting another black box on the end to "fix" the output. This is the opposite of engineering. This is negotiation with a genie trapped in silicon.
it does seem as if the world has gone insane we have brilliant machines that can more or less work perfectly then the scam artists have convinced people that spending a trillion dollar and terawatts to get essentially a biased random number generator to produce unusable garbage is somehow an improvement
I don't know, I think it's pretty cool that we can turn arbitrary human speech into well-formed RPCs.