Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native JSON Output from GPT-4

yonom.substack.com

Post reply on HN
Print view View original

161–170 of 258 posts

Re: Native JSON Output from GPT-4

#161

Post by tornato7 »

brolumir wrote: post #147
bel423 wrote: post #144

Did people really struggle with getting JSON outputs from GPT4. You can literally do it zero shot by just saying match this typescript type. GPT3.5 would output perfect JSON with a single example. I have no idea why people are talking about this like it’s a new development.

Unfortunately, in practice that works only most of the time . At least in our experience (and the article says something similar) sometimes ChatGPT would return something completely different when JSON-formatted response would be expected.

In my experience if you set the temperature to zero it works 99.9% of the time, and then you can just add retry logic for the remaining 0.1%

Re: Native JSON Output from GPT-4

#162

Post by jiggawatts »

m3kw9 wrote: post #87
jonplackett wrote: post #86

This is useful, but for me at least, GPT-4 is unusable because it sometimes takes 30 seconds + to reply to even basic queries.

Also the rate limit is pretty bad if you want to release any type of app

More importantly: there's a waiting list.

Also, if you want to use both the ChatGPT web app and the API, you'll be billed for both separately. They really should be unified and billed under a single account. The difference is literally just whether there's a "web UI" on top of the API... or not.

Re: Native JSON Output from GPT-4

#163

Post by jiggawatts »

chaxor wrote: post #16

Is there a decent way of converting to a structure with a very constrained vocabulary? For example, given some input text, converting it to something like {"OID-189": "QQID-378", "OID-478":"QQID-678"}. Where OID and QQID dictionaries can be e.g. millions of different items defined by a description. The rules for mapping could be essentially what looks closest in semantic space to the descriptions given in a dictionar…

You can train custom GPT 3 models, and Azure now has vector database integration for GPT-based models in the cloud. You can feed it the data, and ask it for the embedding lookup, etc...

You can also host a vector database yourself and fill it up with the embeddings from the OpenAI GPT 3 API.

Re: Native JSON Output from GPT-4

#164

Post by sheepscreek »

jonplackett wrote: post #65
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

The solution that worked great for me - do not use JSON for GPT to agent communication. Use comma separated key=value, or something to that effect.

Then have another pure code layer to parse that into structured JSON.

I think it’s the JSON syntax (with curly braces) that does it in. So YAML or TOML might work just as well, but I haven’t tried that.

Re: Native JSON Output from GPT-4

#165

Post by worik »

bel423 wrote: post #145
ignite wrote: post #106

Earlier quoted context omitted.

> You just append ‘reply in json with this format’ and it does a really good job. It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.

It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.

> It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.

Danger! There be assumptions!!

gpt-? is a moving target and in rapid development. What it does Tuesday, which it did not do on Monday, it may well not do on Wednesday

If there is a documented method to guarantee it, it will work that way (modulo OpenAI bugs - and now Microsoft is involved....)

What we had before, what you are talking of, was observed behaviour. An assumption that what we observed in the past will continue in the future is not something to build a business on

Re: Native JSON Output from GPT-4

#167

Post by worik »

bel423 wrote: post #149
ztratar wrote: post #77

Earlier quoted context omitted.

Nah, this was solved by most teams a while ago.

I feel like I’m taking crazy pills with the amount of people saying this is game changing. Did they not even try asking gpt to format the output as json?

> I feel like I’m taking crazy pills....try asking gpt to format the output as json

You are taking crazey pills. Stop

gpt-? is unreliable! That is not a bug in it, it is the nature of the beast.

It is not an expert at anything except natural language, and even then it is an idiot savant

Re: Native JSON Output from GPT-4

#168

Post by riwsky »

TeMPOraL wrote: post #92
nasir wrote: post #67

Earlier quoted context omitted.

Its a lot more straightforward to use JSON programmatically than YAML.

It really shouldn't be, though. I.e. not unless you're parsing or emitting it ad-hoc, for example by assuming that an expression like: "{" + $someKey + ":" + $someValue + "}" produces a valid JSON. It does - sometimes - and then it's indeed easier to work with. It'll also blow up in your face. Using JSON the right way - via a proper parser and serializer - should be identical to using YAML or any other equivalent for…

Even if the APIs for both were equally simple, modules for manipulating json are way more likely to be available in the stdlib of whatever language you’re using.

Re: Native JSON Output from GPT-4

#169

Post by whateveracct »

bel423 wrote: post #145
ignite wrote: post #106

Earlier quoted context omitted.

> You just append ‘reply in json with this format’ and it does a really good job. It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.

It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.

No it doesn't lol. I've seen it just randomly not use a comma after one array element, for example.

Re: Native JSON Output from GPT-4

#170

Post by whateveracct »

cwxm wrote: post #74
jonplackett wrote: post #65

Earlier quoted context omitted.

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.

It forgets commas too
Post reply on HN