Live data from Hacker News

Show HN: LLMs can generate valid JSON 100% of the time

github.com

281–290 of 315 posts

Re: Show HN: LLMs can generate valid JSON 100% of the time

#281

Hi, remilouf. You say that your background is in "probabilistic, relational and symbolic programming". In that case I suspect you understand that it is no problem to generate text from a regular or context-free grammar, or really any level of grammar. For example, you can do that very easily in Prolog (a relational language) given a grammar in Definite Clause Grammars notation. As far as I can tell your approach requ…

IanCal said it all. But for alternative approaches that also use LLM (with miniKanren) you can check https://arxiv.org/abs/1809.02840

See reply to IanCal's comment then.

Later edit: you have a nice way to generate unstructured text, and now you want to go and bolt a structured representation on top. So now you have to do all the hard work by hand, again, to write the structured representation. That sounds like a regression.

I'll have to make time to read your paper, thanks for linking it.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#282

Hi, remilouf. You say that your background is in "probabilistic, relational and symbolic programming". In that case I suspect you understand that it is no problem to generate text from a regular or context-free grammar, or really any level of grammar. For example, you can do that very easily in Prolog (a relational language) given a grammar in Definite Clause Grammars notation. As far as I can tell your approach requ…

The idea is not to just generate any random string that matches the grammar. The idea is that if your request is "What are the first 10 digits of pi?" and you restrict the response to the regex: "[0-9]+\.[0-9]+", then you actually receive a correct answer of "3.1415926535" and not just a random string such as "1.2346789", which also happens to match the pattern.

That will only work up to the point when the LLM can't generate a correct answer, whether conforming to a grammar or not. After that point, you'll just get grammatically correct bullshit.

Also, as noted in my reply to a sibling comment, grammars do not generate "any random string". That's the whole point of a grammar, that the generation is not random. For example it is perfectly feasible to write a grammar that completes a sentence with missing words, or continues some text etc.

And to be clear, it is entirely feasible to write a grammar that takes some string as input and generates a string as output that is a transformation of the input string satisfying some constraint. This kind of grammar is known as a transducer.

None of this should come as a surprise. Statistical language models are simply an alternative to knowledge-engineered grammars, used to do the same things that one can do with a grammar (except for the determinism). In a broad sense, a statistical language model is a kind of grammar, or perhaps it makes more sense to say that a grammar is a deterministic language model.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#283
post #247

Earlier quoted context omitted.

Wouldn't that generate an entirely random but valid output? Here you want a valid output related to the request. > And why would you need an LLM, a model of natural language, if all you want is to generate structured text, anyway? So that you can parse unstructured text from a person and return structured data for a machine.

>> Wouldn't that generate an entirely random but valid output? No. Grammars don't generate entirely random output. Even Probabilistic Context Free Grammars can generate deterministic output, depending on how they are sampled. The output can be related to some input, if desired, for example one can give a string with "holes" (variables) as input and have the holes filled-in by the grammar. >> So that you can parse uns…

I wasn't talking about deterministic Vs nondeterministic.

> If you are willing to spend the effort to write a grammar, you can do that without an LLM.

How are you taking, for example, a request to make a "fun but not over the top character from the middle ages, with relevant weapons and a backstory. Game theme is a world populated by anthropomorphic vegetables." And get back a character for the game in a specific JSON format without the LLM in your design here? That's not encodable in the grammar.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#284
post #233

A major part of the power of an LLM is the calibrated probability distribution in its responses, and this technique probably throws that ability away. Why is it good enough? As a brief example, suppose the only possible LLM outputs were "hello world", "food", "hello", and "good day" (and that they're all equally probable with no prompting). Suppose your grammar requires a space in the output somewhere and has no othe…

In this case (multiple choice generation), if one of the possible outputs does no match the regex, you can just exclude it from generation. I am trying to think of an example where "answer prefix might have been extremely unlikely to yield a valid response, but the technique ( ... ) constructs a valid response from it regardless" , which might really cause a problem. But to no luck. Anyone has any idea? This could po…

The multiple choice example was just for tractable computations and illustrative purposes. Pretend the LLM has characters===tokens and is doing autoregressive probability prediction as per usual -- "f"-25%, "h"-50%, "g"-25% to start with, and then appropriate probabilities thereafter to yield that multiple-choice example (plus an token).

> I am trying to think of an example where "answer prefix might have been extremely unlikely to yield a valid response, but the technique ( ... ) constructs a valid response from it regardless", which might really cause a problem. But to no luck. Anyone has any idea? This could potentially be an interesting research question.

At one point in the past ChatGPT (at a model probability layer, not just because of the context window issue) was prone to truncating long JSON responses, and if that happened in a long string field then you'd see the observed behavior. An example application:

(-) You're asking the LLM to turn some written podcast description into something machine-readable. You chunk the input, feed each chunk into the model (somehow; ignore the details; they're not important), and turn paragraphs into {speaker_name: str, timestamp: str, content: str} blobs.

(1) The LLM is prone to turning long paragraphs into `{"content": "the beginning of the content...` patterns, using ellipses to indicate that there's more to that JSON object.

(2) If you actually retry till the LLM succeeds, it's leaps and bounds more likely to end that string with a quotation mark if the string has all the original input. I.e., output like `{"content": "the beginning of the content..."}` is comparatively rare.

(3) The article's technique, however, always morphs those truncated json blobs into valid json. Since the ellipses is _valid_ at that point (a sub-string), instead of the vast majority of inputs failing you instead end up with the vast majority succeeding and having an incorrect ellipses sub-string.

In general, the LLM does autoregressive completions. Imagine two prefixes P1 and P2, each of which can be completed by classes of data so that P1{G1} adheres to the grammar, P1{F1} fails to adhere to the grammar, P2{G2} succeeds, and P2{F2} fails. With retry-till-passing-grammar the weighted probabilities are:

P1{G1}: Chance[P1] Chance[G1 | P1]

P2{G2}: Chance[P2] Chance[G2 | P2]

Whereas the weighted probabilities produced by the technique are:

P1{G1}: Chance[P1]

P2{G2}: Chance[P2]

In both cases you'd need to divide by the total probability, but the convolution by conditionals is both important and notably absent. For very simple schemas like {sentiment: "positive"|"negative"|"neutral"} the results might potentially be similar, but nothing in the idea of a greedy token filter forces that constraint.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#285
post #283

Earlier quoted context omitted.

>> Wouldn't that generate an entirely random but valid output? No. Grammars don't generate entirely random output. Even Probabilistic Context Free Grammars can generate deterministic output, depending on how they are sampled. The output can be related to some input, if desired, for example one can give a string with "holes" (variables) as input and have the holes filled-in by the grammar. >> So that you can parse uns…

I wasn't talking about deterministic Vs nondeterministic. > If you are willing to spend the effort to write a grammar, you can do that without an LLM. How are you taking, for example, a request to make a "fun but not over the top character from the middle ages, with relevant weapons and a backstory. Game theme is a world populated by anthropomorphic vegetables." And get back a character for the game in a specific JSO…

As far as I can tell you won't be able to use the approach proposed here to create a character matching your above description unless every element of it is encoded in the guiding grammar (including the possibility for the character to have middle ages-relevant weapons, and the anthropomorphic vegetables).

At which point, again I have to ask: what do you need the LLM for? You've already done all the hard work by hand and the LLM is only adding some extraneous natural language parsing on top.

Plus, if you already have the grammar that can cover the anthropomorphic vegetable world it's only a bit more work to use it to parse such natural language requests, anyway.

I think people forget that grammars were the staple for parsing natural language and stuffing it into structured form for a very long time before LLMs, and they still mostly are.

The point is that if you have structure, someone has to hand-craft that structure. Frex, if you have a language with a compiler, someone has to write the compiler. Then, if you want to make some unstructured text conform to your hand-crafted structure, you can only do that to the extent that the unstructured text itself is made up of elements of the structured form. If you have a grammar for frogs and blueberries, and write a poem about the dawn and foxes, you can't use the former to structure the latter, no matter what you do, and LLMs won't make this happen magickally, either.

Essentially, your grammar is a type and any unstructured text you want to convert to a structure with your grammar must be a value that you can cast to that type.

>> I wasn't talking about deterministic Vs nondeterministic.

Then what? What do you mean by "random string"?

Re: Show HN: LLMs can generate valid JSON 100% of the time

#286

I have noob thought on the potential of these in Formal path planning. Specifically given a set of functions that basically map {State -> Actions} given preconditions, transition functions (heavily paraphrasing STRIPS[1]) can a correct and optionally "realistic" plan be generated[2]? I am quite interested in this. It seems clear that the issue is that there is no "guidance" like DFA on what is the correct next symbol…

[deleted]

Re: Show HN: LLMs can generate valid JSON 100% of the time

#287

I have noob thought on the potential of these in Formal path planning. Specifically given a set of functions that basically map {State -> Actions} given preconditions, transition functions (heavily paraphrasing STRIPS[1]) can a correct and optionally "realistic" plan be generated[2]? I am quite interested in this. It seems clear that the issue is that there is no "guidance" like DFA on what is the correct next symbol…

>> Specifically given a set of functions that basically map {State -> Actions} given preconditions, transition functions (heavily paraphrasing STRIPS[1]) can a correct and optionally "realistic" plan be generated[2]?

Maybe, but the results would be unreliable. And if there's one thing that Good, Old-Fashioned, automated planning and scheduling is good at, that is reliability.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#288
post #103

I really hope OpenAI add something like this to their endpoints soon. Being able to pass up some kind of grammar (a regular expression, or a JSON schema, or some other format) and have this trick run during their token sampling process to ensure the output was compliant would be incredibly useful.

They recently added logit biases, so that's a start.

It's limited to 300 logit biases at a time. Knowing GPT4's vocabulary is ~100k tokens it's not nearly enough to get reliable guided generation. Although it could work in some cases, and another advantage of this work is that we can determine that before generating.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#289

Mechanistically, I think this library takes the simple idea of masking part of the vocabulary space and steps in time efficiently. Great! I am curious, however, for the ones who have played around with such libraries wrapping base LLMs with output structure: do base models like Llama2 work very well? My experience says "hell no!" and you do need a fair bit of instruction-tuning for specific use cases to actually get…

In our experience, at least for code generation, the experience has been that base models can be improved significantly by guiding token level generation. In our paper titled "Guiding Language Models of Code with Global Context using Monitors" ( https://arxiv.org/abs/2306.10763 ), we propose Monitor Guided Decoding, which interfaces LLMs to static analysis, and guides the model to generate type-consistent code. Witho…

Thanks for the reference, Lakshya. Looks very cool!

(Just thinking out loud next)

If you allow me to be a little imprecise, guided-generation is prompting "just-in-time" unlike the other kind of prompting where you provide all reference tokens "ahead-of-time". Now there's work [1] out there that shows that smaller models rely much more on prompting than larger models do, i.e. smaller models are more faithful to the tokens in the prompt than the larger models which just do whatever they were going to do anyways.

Your results seem very much in line with this kind of a qualitative result --- you show that CodeGen-350M outperforms CodeGen-6B, and CodeGen-6B outperforms text-davinci-003 using MGD. Smaller models perhaps respond more strongly to certain kinds of prompting strategies than larger models do.

[1]: https://arxiv.org/pdf/2307.13702.pdf

Re: Show HN: LLMs can generate valid JSON 100% of the time

#290
post #209

Earlier quoted context omitted.

it would also be nice to see one example that uses gpt4.

Given how this works, I don’t think that is possible unless OpenAI implements it themselves.

really? the docs seem to promise something like that "can work with any model"
Post reply on HN