Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native JSON Output from GPT-4

yonom.substack.com

Post reply on HN
Print view View original

131–140 of 258 posts

Re: Native JSON Output from GPT-4

#131

Post by [deleted] »

ignite wrote: post #106
jonplackett wrote: post #65

Earlier quoted context omitted.

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

> You just append ‘reply in json with this format’ and it does a really good job. It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.

[deleted]

Re: Native JSON Output from GPT-4

#132

Post by sethd »

jonplackett wrote: post #65
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

I like to define a JSON schema (https://json-schema.org/) and prompt GPT-4 to output JSON based on that schema.

This lets me specify general requirements (not just JSON structure) inline with the schema and in a very detailed and structured manor.

Re: Native JSON Output from GPT-4

#133

Post by memefrog »

TeMPOraL wrote: post #108
moneywoes wrote: post #97

Earlier quoted context omitted.

Wow your brand is huge. Crazy growth. i wonder how much these subtle mentions on forums help

They're the only one commenter on HN I noticed keeps writing "smol" instead of "small", and is associated with projects with "smol" in their name. Surely I'm not the only one who missed it being a meme around 2015 or sth., and finds this word/use jarring - and therefore very attention-grabbing? Wonder how much that helps with marketing. This is meant with no negative intentions. It's just that 'swyx was, in my mind,…

Personally, I associate "smol" with "doggo" and "chonker" and other childish redditspeak.

Re: Native JSON Output from GPT-4

#134

Post by fumar »

majormajor wrote: post #28
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

GPT-4 was already a massive improvement on 3.5 in terms of replying consistently in a certain JSON structure - I often don't even need to give examples, just a sentence describing the format. It's great to see they're making it even better, but where I'm currently hitting the limit still in GPT-4 for "shelling out" is about it being truly "creative" or "introspective" about "do I need to ask for clarifications" or "c…

It is “good enough”. Where I struggle is maintaining its memory through a longer request where multiple iterations fail or succeed and then all of a sudden its memory is exceeded and starts fresh. I wish I could store “learnings” that it could revisit.

Re: Native JSON Output from GPT-4

#135

Post by taylorfinley »

bradly wrote: post #125
rytill wrote: post #100

Earlier quoted context omitted.

Did you try using the API and providing a very clear system message followed by several examples that were pure JSON?

Yep. I even gave it a JSON schema file to use. It just wouldn't stop added extra verbage.

I just use a regex to select everything between the first and last curly bracket, reliable fixes the “sure, here’s your object” problem.

Re: Native JSON Output from GPT-4

#136

Post by girvo »

8organicbits wrote: post #121
jonplackett wrote: post #85

Earlier quoted context omitted.

Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

Wouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?

In my experience, telling it "no thats wrong, try again" just gets it to be wrong in a new different way, or restate the same wrong answer slightly differently. I've had to explicitly guide it to correct answers or formats at times.

Re: Native JSON Output from GPT-4

#137

Post by seizethecheese »

jonplackett wrote: post #65
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

In a production system, you don’t need easy to do most of the time, you need easy without fail.

Re: Native JSON Output from GPT-4

#138

Post by pnpnp »

seizethecheese wrote: post #137
jonplackett wrote: post #65

Earlier quoted context omitted.

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

In a production system, you don’t need easy to do most of the time, you need easy without fail.

Ok, just playing devil's advocate here. How many FAANG companies have you seen have an outage this year? What's their budget?

I think a better way to reply to the author would have been "how often does it fail"?

Every system will have outages, it's just a matter of how much money you can throw at the problem to reduce them.

Re: Native JSON Output from GPT-4

#139

Post by jrockway »

pnpnp wrote: post #138
seizethecheese wrote: post #137

Earlier quoted context omitted.

In a production system, you don’t need easy to do most of the time, you need easy without fail.

Ok, just playing devil's advocate here. How many FAANG companies have you seen have an outage this year? What's their budget? I think a better way to reply to the author would have been "how often does it fail"? Every system will have outages, it's just a matter of how much money you can throw at the problem to reduce them.

If 99.995% correct looks bad to users, wait until they see 37%.
Post reply on HN