Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native JSON Output from GPT-4

yonom.substack.com

Post reply on HN
Print view View original

151–160 of 258 posts

Re: Native JSON Output from GPT-4

#151

Post by bel423 »

freezed88 wrote: post #129
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

100%, if the API itself can choose to call a function or an LLM, then it's way easier to build any agent loop without extensive prompt engineering + worrying about errors. Tweeted about it here as well: https://twitter.com/jerryjliu0/status/1668994580396621827?s=...

You still have to worry about errors. You will probably have to add an error handler function that it can call out to. Otherwise the LLM will hallucinate a valid output regardless of the input. You want it to be able to throw an error and say I could produce the output given this format.

Re: Native JSON Output from GPT-4

#152

Post by dror »

bradly wrote: post #140
NicoJuicy wrote: post #130

Earlier quoted context omitted.

Say it's a json API and may only reply with valid json without explanation.

Lol yes of course I tried that.

I've had good luck with both:

https://github.com/drorm/gish/blob/main/tasks/coding.txt

and

https://github.com/drorm/gish/blob/main/tasks/webapp.txt

With the second one, I reliably generated half a dozen apps with one command.

Not to say that it won't fail sometimes.

Re: Native JSON Output from GPT-4

#153

Post by killingtime74 »

swyx wrote: post #150
killingtime74 wrote: post #148

Earlier quoted context omitted.

Who is Simon Willison? Is he big in AI?

formerly cocreator of Django, now Datasette, but pretty much the top writer/hacker on HN making AI topics accessible to engineers https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...

Oh wow, nice! Big fan of his work

Re: Native JSON Output from GPT-4

#154

Post by muzani »

jonplackett wrote: post #65
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

It's fine, but the article makes some good points why - less cognitive load for GPT and less tokens. I think the transistor to logic gate analogy makes sense. You can build the thing perfectly with transistors, but just use the logic gate lol.

Re: Native JSON Output from GPT-4

#155

Post by ehsanu1 »

fumar wrote: post #134
majormajor wrote: post #28

Earlier quoted context omitted.

GPT-4 was already a massive improvement on 3.5 in terms of replying consistently in a certain JSON structure - I often don't even need to give examples, just a sentence describing the format. It's great to see they're making it even better, but where I'm currently hitting the limit still in GPT-4 for "shelling out" is about it being truly "creative" or "introspective" about "do I need to ask for clarifications" or "c…

It is “good enough”. Where I struggle is maintaining its memory through a longer request where multiple iterations fail or succeed and then all of a sudden its memory is exceeded and starts fresh. I wish I could store “learnings” that it could revisit.

Sounds like you want something like tree of thoughts: https://arxiv.org/abs/2305.10601

Re: Native JSON Output from GPT-4

#156

Post by blamy »

thorum wrote: post #2

The JSON schema not counting toward token usage is huge, that will really help reduce costs.

But it does count toward token usage. And they picked JSON schema which is like 6x more verbose than typescript for defining the shape of json.

Re: Native JSON Output from GPT-4

#157

Post by blamy »

brolumir wrote: post #147
bel423 wrote: post #144

Did people really struggle with getting JSON outputs from GPT4. You can literally do it zero shot by just saying match this typescript type. GPT3.5 would output perfect JSON with a single example. I have no idea why people are talking about this like it’s a new development.

Unfortunately, in practice that works only most of the time . At least in our experience (and the article says something similar) sometimes ChatGPT would return something completely different when JSON-formatted response would be expected.

I've been using the same prompts for months and have never seen this happen on 3.5-turbo let alone 4.

https://gist.github.com/BLamy/244eec016beb9ad8ed48cf61fd2054...

Re: Native JSON Output from GPT-4

#158

Post by swyx »

babyshake wrote: post #101
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

What would be an example where there needs to be an arbitrary level of recursive ability for GPT4 to call itself?

writing code of higher complexity (we know from CICERO that longer time spent on inference is worth orders of magnitude more than the equivalent in training when it comes to improving end performance), or doing real world tasks with unknown fractal depth (aka yak shave)

Re: Native JSON Output from GPT-4

#160

Post by swyx »

mritchie712 wrote: post #83
swyx wrote: post #11

Earlier quoted context omitted.

i mean only at the most superficial level. she has a ton of other validators that arent superceded (eg SQL is validated by branching the database - we discussed on our pod https://www.latent.space/p/guaranteed-quality-and-structure )

yeah, listened to the pod (that's how I found out about guardrails!). fair point, I should have said: "value prop for our use case"... the thing I was most interested in was how well Guardrails structured output.

haha excellent. i was quite impressed by her and the vision for guardrails. thanks for listening!
Post reply on HN