Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: LLMs can generate valid JSON 100% of the time

github.com

Post reply on HN
Print view View original

171–180 of 315 posts

Re: Show HN: LLMs can generate valid JSON 100% of the time

#171

Post by frontier »

OJFord wrote: post #160
quickthrower2 wrote: post #130

> LLMs can generate valid JSON 100% of the time If that seems surprising, it is worth doing a course like Karpathy's zero to hero NN, and have all the magic peeled away a layer at a time. The reason you can do this is because LLMs don't just generate the next word or token, it produces a probability distribution over all tokens. A JSON parser can give you a list of next valid tokens. The tokens in each case might be…

Maybe it's just me, but I'm not doing anything that calls itself 'zero to hero'. Would love some good resources (preferably textbook, or at least written) on LLMs though. I don't even understand the link to 'generative' image/video AI, which seems to have exploded at roughly the same time and surely isn't a coincidence. I studied a little (literally 'intro to') ML at university, about enough to grok it as an applicat…

Karpathy's series is many many hours long and really does take you from zero to GPT. It's excellent! You sound triggered by the title - that may not even be the official title - but it definitely deserves it. Go look it up.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#172

Post by selcuka »

caesil wrote: post #104
panarky wrote: post #8

I can make GPT4 return valid JSON simply by providing examples in the system message. This works nine times out of ten. But it's still probabilistic, and nine times out of ten isn't good enough. Occasionally it will hallucinate responses like this: {"key1": "value1", "key2": "value2" for i in range(n)} Re-prompting with the parsing error message is usually enough to get it on the second try. But escaping double-quote…

With ChatGPT function calling I get valid JSON 100% of the time from GPT-4 unless I have made some error in prompting. The chief error is not providing escape hatches. LLMs look for a right answer. If you are feeding it some texts and asking it to return structured data about the texts, but then one of the texts is blank, it will be difficult to determine a right answer, so you get hallucinations. The solution is an…

The premise of function calling is great, but in my experience (at least on GPT-3.5, haven't tried it with GPT-4 yet) it seems to generate wildly different, and less useful results, for the same prompt.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#174

Post by antonvs »

neoncontrails wrote: post #168
dwattttt wrote: post #139

Earlier quoted context omitted.

If you're choosing the next token based on a list of valid next tokens, a uniform random distribution can always generate valid JSON too!

But that's not what an LLM does.

The point is that if you're "choosing the next token based on a list of valid next tokens," it's not surprising that you'll only generate valid output, since absolutely any choice mechanism will suffice.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#175

Post by antonvs »

OJFord wrote: post #160
quickthrower2 wrote: post #130

> LLMs can generate valid JSON 100% of the time If that seems surprising, it is worth doing a course like Karpathy's zero to hero NN, and have all the magic peeled away a layer at a time. The reason you can do this is because LLMs don't just generate the next word or token, it produces a probability distribution over all tokens. A JSON parser can give you a list of next valid tokens. The tokens in each case might be…

Maybe it's just me, but I'm not doing anything that calls itself 'zero to hero'. Would love some good resources (preferably textbook, or at least written) on LLMs though. I don't even understand the link to 'generative' image/video AI, which seems to have exploded at roughly the same time and surely isn't a coincidence. I studied a little (literally 'intro to') ML at university, about enough to grok it as an applicat…

> I'm not doing anything that calls itself 'zero to hero'.

Sounds like you have a case of the Mondays. You just need to turn that frown upside down!

Re: Show HN: LLMs can generate valid JSON 100% of the time

#176

Post by OJFord »

frontier wrote: post #171
OJFord wrote: post #160

Earlier quoted context omitted.

Maybe it's just me, but I'm not doing anything that calls itself 'zero to hero'. Would love some good resources (preferably textbook, or at least written) on LLMs though. I don't even understand the link to 'generative' image/video AI, which seems to have exploded at roughly the same time and surely isn't a coincidence. I studied a little (literally 'intro to') ML at university, about enough to grok it as an applicat…

Karpathy's series is many many hours long and really does take you from zero to GPT. It's excellent! You sound triggered by the title - that may not even be the official title - but it definitely deserves it. Go look it up.

The title suggests I wouldn't like it, yes. But as a video series it's not 'a textbook or at least written' is i - not really the format I'm looking for personally.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#177

Post by antonvs »

catlifeonmars wrote: post #167
padjo wrote: post #97

Earlier quoted context omitted.

It’s starting to feel like LLMs are to “classical” software engineering what quantum physics was to classical physics

How so? I’m not quite following the analogy.

Just guessing what was meant, but quantum physics in some sense tries all possible paths before an outcome is selected.

The problem with that is that without a quantum computer, or without some sort of filtering, that process can take up to infinite time.

Re: Show HN: LLMs can generate valid JSON 100% of the time

#178

Post by thatcherthorn »

OJFord wrote: post #176
frontier wrote: post #171

Earlier quoted context omitted.

Karpathy's series is many many hours long and really does take you from zero to GPT. It's excellent! You sound triggered by the title - that may not even be the official title - but it definitely deserves it. Go look it up.

The title suggests I wouldn't like it, yes. But as a video series it's not 'a textbook or at least written' is i - not really the format I'm looking for personally.

Truly.. one of the greatest minds in our ML era. Don't get caught up on the format :)

Re: Show HN: LLMs can generate valid JSON 100% of the time

#179

Post by dilawar »

mattigames wrote: post #135
quickthrower2 wrote: post #130

> LLMs can generate valid JSON 100% of the time If that seems surprising, it is worth doing a course like Karpathy's zero to hero NN, and have all the magic peeled away a layer at a time. The reason you can do this is because LLMs don't just generate the next word or token, it produces a probability distribution over all tokens. A JSON parser can give you a list of next valid tokens. The tokens in each case might be…

It's not like humans are particularly good at distinguishing truth from lies.

"Its a human nature to mislead others, sometimes knowingly." I read this line in an anthropology book. A similarly non-cynical approach towards your fellow is "trust but verify".

Re: Show HN: LLMs can generate valid JSON 100% of the time

#180

Post by [deleted] »

quickthrower2 wrote: post #130

> LLMs can generate valid JSON 100% of the time If that seems surprising, it is worth doing a course like Karpathy's zero to hero NN, and have all the magic peeled away a layer at a time. The reason you can do this is because LLMs don't just generate the next word or token, it produces a probability distribution over all tokens. A JSON parser can give you a list of next valid tokens. The tokens in each case might be…

[deleted]
Post reply on HN