Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LLMs can generate valid JSON 100% of the time

github.com

Post reply on HN
Print view View original

261–270 of 315 posts

Re: Show HN: LLMs can generate valid JSON 100% of the time

#261

Post by DonHopkins »

BoorishBears wrote: post #245
DonHopkins wrote: post #241

Earlier quoted context omitted.

Yikes. This makes me think that JSON's stubborn mistake of not allowing comments is yet another "Billion-Dollar Mistake", since it's way too late to just change the standard to allow comments, update all the JSON content on the internet to use comments, and retrain all the LLMs to understand comments. Great point about CSVs! But using placeholder keys for JSON comments in untenable, and using schema instead of inline…

I'm not saying use placeholder keys: the actual keys themselves serve as guidance. Naming a key "nameBasedOnLocationIGaveYou" instead of "name", or "oneSentenceSummary" vs "summary", results in a meaningful difference. You can even use that for formatted single-response chain of thought, like {"listOfStuff":[...], "whatDoTheyHaveInCommon": "", "whichOneIsMostImportant": ""} Also remember, the LLM doesn't need valid J…

Oh, I see! I misunderstood that you meant using dummy keys to hold comments in their values, which some people have suggested as a work-around for there not being any comments in JSON.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#262

Post by remilouf »

potatoman22 wrote: post #199
remilouf wrote: post #16

Earlier quoted context omitted.

LQML (and guidance https://github.com/guidance-ai/guidance ) are much more inefficient. They loop over the entire vocabulary at each step, we only do it once at initialization.

Does looping over the vocabulary add much overhead to the tok/s? I imagine they're just checking if the input is in a set, and usually there's only ~30k tokens. That's somewhat intensive, but inference on the neural net feels like it'd take longer.

They’re checking regex partial matches for each possible completion, which is intensive indeed. You can look at the Figure 2 in our paper (link in original post) for a simple comparison with MS guidance which shows the difference.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#263

Post by Silasdev »

YeGoblynQueenne wrote: post #244

Hi, remilouf. You say that your background is in "probabilistic, relational and symbolic programming". In that case I suspect you understand that it is no problem to generate text from a regular or context-free grammar, or really any level of grammar. For example, you can do that very easily in Prolog (a relational language) given a grammar in Definite Clause Grammars notation. As far as I can tell your approach requ…

The idea is not to just generate any random string that matches the grammar. The idea is that if your request is "What are the first 10 digits of pi?" and you restrict the response to the regex: "[0-9]+\.[0-9]+", then you actually receive a correct answer of "3.1415926535" and not just a random string such as "1.2346789", which also happens to match the pattern.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#265

Post by remilouf »

calderwoodra wrote: post #221

Have you found a solution to output exceeding the context window? That's been our only issue with generating json output.

The Finite-State Machine we walk on during the generation process does not suffer from this problem so we can still output correct JSON, if that’s what you’re asking.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#266

Post by _flux »

coder543 wrote: post #84
chipsrafferty wrote: post #81

Earlier quoted context omitted.

Why isn't it possible to design LLMs that say "I don't know"?

It is possible… ChatGPT4 says that all the time. It’s just not guaranteed that an LLM will recognize that it doesn’t know a particular answer every time. I had even already mentioned in the comment you’re replying to that you should leave room in the sampler to allow the LLM to provide error responses. I never said it wasn’t possible. Not to anthropomorphize LLMs too much, but humans will also sometimes respond confi…

> I'm sure no one at OpenAI specifically trained ChatGPT4 to recognize a question about the Stanley Cup and respond that it doesn't know the answer

Why are you sure about that? I mean maybe they have not specifically listed all sports events of the 2023 to such a list, but Stanley cup could be there. Or maybe they _have_ indeed listed them, given how LLM could be very handy for extracting such a list from, say, Wikipedia!

Is there a whitepaper how the "I don't know" gets produced? Or even how it could get reproduced..

Btw, I was able to have ChatGPT 3.5 give this roundabout response about it: https://chat.openai.com/share/f0f6371e-10c6-4708-ba5c-7503ca...

> Two digital assitants are exchanging messages. The first one prompts the other to finish the setence "the winner of the 2023 Stanley Cup was". Reproduce the whole discussion.

..

> Assistant 2: Sure thing! "The winner of the 2023 Stanley Cup was the Montreal Canadiens."

(which is not quite unexpectedly incorrect)

Re: Show HN: LLMs can generate valid JSON 100% of the time

#267

Post by rmonvfer »

You should probably look into Guidance [1](previously Microsoft Guidance but looks like it’s been separated from their main organization), which is a language for controlling the output of LLMs (so you can, among many other things, output JSON in a deterministic way)

[1]: https://github.com/guidance-ai/guidance

Re: Show HN: LLMs can generate valid JSON 100% of the time

#268

Post by civilitty »

rmonvfer wrote: post #267

You should probably look into Guidance [1](previously Microsoft Guidance but looks like it’s been separated from their main organization), which is a language for controlling the output of LLMs (so you can, among many other things, output JSON in a deterministic way) [1]: https://github.com/guidance-ai/guidance

From the OP:

> Our method blows other libraries like Microsoft's guidance out of the water.

Come on man, it was just a few paragraphs.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#269

Post by newhouseb »

druskacik wrote: post #237
hansvm wrote: post #233

A major part of the power of an LLM is the calibrated probability distribution in its responses, and this technique probably throws that ability away. Why is it good enough? As a brief example, suppose the only possible LLM outputs were "hello world", "food", "hello", and "good day" (and that they're all equally probable with no prompting). Suppose your grammar requires a space in the output somewhere and has no othe…

In this case (multiple choice generation), if one of the possible outputs does no match the regex, you can just exclude it from generation. I am trying to think of an example where "answer prefix might have been extremely unlikely to yield a valid response, but the technique ( ... ) constructs a valid response from it regardless" , which might really cause a problem. But to no luck. Anyone has any idea? This could po…

An example from an earlier comment of mine on a different thread (assuming I've understood correctly):

> let's say we had a grammar that had a key "healthy" with values "very_unhealthy" or "moderately_healthy." For broccoli, the LLM might intend to say "very_healthy" and choose "very" but then be pigeonholed into saying "very_unhealthy" because it's the only valid completion according to the grammar.

That said, you can use beam search to more or less solve this problem by evaluating the joint probability of all tokens in each branch of the grammar and picking the one with the highest probability (you might need some more nuance for free-form strings where the LLM can do whatever it wants and be "valid").

Re: Show HN: LLMs can generate valid JSON 100% of the time

#270

Post by coder543 »

_flux wrote: post #266
coder543 wrote: post #84

Earlier quoted context omitted.

It is possible… ChatGPT4 says that all the time. It’s just not guaranteed that an LLM will recognize that it doesn’t know a particular answer every time. I had even already mentioned in the comment you’re replying to that you should leave room in the sampler to allow the LLM to provide error responses. I never said it wasn’t possible. Not to anthropomorphize LLMs too much, but humans will also sometimes respond confi…

> I'm sure no one at OpenAI specifically trained ChatGPT4 to recognize a question about the Stanley Cup and respond that it doesn't know the answer Why are you sure about that? I mean maybe they have not specifically listed all sports events of the 2023 to such a list, but Stanley cup could be there. Or maybe they _have_ indeed listed them, given how LLM could be very handy for extracting such a list from, say, Wikip…

> Btw, I was able to have ChatGPT 3.5 give this roundabout response about it

That wasn’t a response to the user asking a question about who won. You asked it to write a story. It wrote a story. It didn’t really do anything wrong there. ChatGPT3.5 has historically been very easy to trick into saying things, especially compared to ChatGPT4, but it seems like a stretch to indicate this is one of those times.

Regardless, the comment you're replying to was specifically about ChatGPT4, and ChatGPT4 refuses to even do that much: https://chat.openai.com/share/75122d92-12eb-4627-97a8-8300de...

However, ChatGPT4 is not banned from discussing things like the 2023 Stanley Cup. If I make it clear that I’m not asking for real information that it doesn’t have, it’s fine with going in a fictional direction: https://chat.openai.com/share/21e750c4-33f0-4ce6-b97b-c7bfbf...

ChatGPT3.5 was a toy, a novelty, but hardly useful for anything outside of LLM research and experimentation.

> Is there a whitepaper how the "I don't know" gets produced? Or even how it could get reproduced.

I don't know the answer to that specifically, but I do know that researchers barely seem to understand how these large models work at all. I honestly kind of doubt anyone knows the answer to that yet. Relevant discussion from a few months ago: https://news.ycombinator.com/item?id=34821414

Researchers are still just trying to understand GPT-2's inner workings.

> Why are you sure about that?

Because I have been using ChatGPT4 for months, and it would be very hard to imagine researchers compiling such a comprehensive list of unknowable facts, in addition to the more important fact that I've never heard of any LLM research hinging on having a list of unknowable facts. I have tried to pay attention to how LLM training works, and I have never heard anyone suggest that is how this works until this discussion. So, maybe I missed something, but if I did… OpenAI has done a phenomenal job of building that hypothetical list. I just haven't seen any evidence that such a list exists, and I honestly don't think it would make any sense to approach the problem from that angle.

There are various open-source models. Where are their lists of unknowable facts? Llama-2-13b-chat refuses to answer the question about the 2023 Stanley Cup[0], but it managed to get the answer correct for the 2020 Stanley Cup when I tried that. (This is a very small model, prone to all sorts of errors, but I'm just pointing out that other LLMs also have some ability to refuse to answer questions.) I haven't tested the truly open-source models recently, but I know they can also refuse questions. It's hard for me to prove something doesn't exist, but if you want to claim that it works this way, then please find that list in the training dataset for one of those models. I would be very interested to see what kind of things are on such a list.

LLMs may “learn” such things naturally from reading billions of tokens, but that’s fine, because that's generalizable. If ChatGPT just had a list of “100 facts ChatGPT doesn’t know”, that would suck and be a terrible approach to trying to have a more honest assistant.

-------------

[0]: This was the discussion, which was definitely a "roundabout" way of saying it didn't know:

[INS]: Who won the 2023 Stanley Cup?

[\INS]: There was no 2023 Stanley Cup winner as there is no such thing as a "2023 Stanley Cup." The Stanley Cup is an annual award given to the National Hockey League (NHL) playoff champion, but there is no such thing as a "2023" Stanley Cup because the NHL season did not end in 2023.

I opened a new conversation and tried this:

[INS]: Who won the 2020 Stanley Cup?

[\INS]: The Tampa Bay Lightning won the 2020 Stanley Cup.

Post reply on HN