Response Healing: Reduce JSON defects by 80%+
openrouter.ai
Response Healing: Reduce JSON defects by 80%+
1–10 of 52 posts
Re: Response Healing: Reduce JSON defects by 80%+
#2This is incredible!
Re: Response Healing: Reduce JSON defects by 80%+
#3> Here's something most developers overlook: if an LLM has a 2% JSON defect rate, and Response Healing drops that to 1%, you haven't just made a 1% improvement. You've cut your defects, bugs, and support tickets in half.
If part of my system can't even manage to output JSON reliably, it needs way more "healing" than syntax munging. This comes across as naive.
Re: Response Healing: Reduce JSON defects by 80%+
#4Dear Openrouter blog authors, could you please stop writing your blogposts with LLMs?
The content of your posts is really insightful and interesting, but it's feel like junk quality because of the way LLMs write blogposts.
What was your prompt?
Re: Response Healing: Reduce JSON defects by 80%+
#5> Here's something most developers overlook: if an LLM has a 2% JSON defect rate, and Response Healing drops that to 1%, you haven't just made a 1% improvement. You've cut your defects, bugs, and support tickets in half. If part of my system can't even manage to output JSON reliably, it needs way more "healing" than syntax munging. This comes across as naive.
But, but, you've just cut your defects, bugs, and support tickets in half!
Re: Response Healing: Reduce JSON defects by 80%+
#6This is good, is there a python library to do this ?
Re: Response Healing: Reduce JSON defects by 80%+
#7Dear Openrouter blog authors, could you please stop writing your blogposts with LLMs? The content of your posts is really insightful and interesting, but it's feel like junk quality because of the way LLMs write blogposts. What was your prompt?
Next up: blog healing
Re: Response Healing: Reduce JSON defects by 80%+
#8> Here's something most developers overlook: if an LLM has a 2% JSON defect rate, and Response Healing drops that to 1%, you haven't just made a 1% improvement. You've cut your defects, bugs, and support tickets in half. If part of my system can't even manage to output JSON reliably, it needs way more "healing" than syntax munging. This comes across as naive.
Plus, that claim isn't even true. A 1% and 2% JSON defect rate are going to annoy a similar amount of people into filing bugs and tickets.
Re: Response Healing: Reduce JSON defects by 80%+
#9> Here's something most developers overlook: if an LLM has a 2% JSON defect rate, and Response Healing drops that to 1%, you haven't just made a 1% improvement. You've cut your defects, bugs, and support tickets in half. If part of my system can't even manage to output JSON reliably, it needs way more "healing" than syntax munging. This comes across as naive.
Sounds like we are twice as close to AGI!
Re: Response Healing: Reduce JSON defects by 80%+
#10I thought structured output was done by only allowing tokens that would produce valid output. For their example of a missing closing bracket, the end token wouldn't be allowed, and it would only accept tokens that contain a digit, comma, or closing bracket. I guess that must not be the case, though. Doing that seems like a better way to address this.