Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native JSON Output from GPT-4

yonom.substack.com

Post reply on HN
Print view View original

101–110 of 258 posts

Re: Native JSON Output from GPT-4

#101

Post by babyshake »

swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

What would be an example where there needs to be an arbitrary level of recursive ability for GPT4 to call itself?

Re: Native JSON Output from GPT-4

#102

Post by SamPatt »

jonplackett wrote: post #85
cwxm wrote: post #74

Earlier quoted context omitted.

even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.

Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.

Re: Native JSON Output from GPT-4

#103

Post by naiv »

ilaksh wrote: post #80
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

The thing is the relevant context often depends on what it's trying to do. You can give it a lot of context in 16k but if there are too many different types of things then I think it will be confused or at least have less capacity for the actual selected task. So what I am thinking is that some functions might just be like gateways into a second menu level. So instead of just edit_file with the filename and new sourc…

I think big context only makes sense for document analysis.

For programming you want to keep it slim. Just like you should keep your controllers and classes slim.

Also people with 32k access report very very long response times of up to multiple minutes which is not feasible if you only want a smaller change or analysis.

Re: Native JSON Output from GPT-4

#104

Post by jonplackett »

SamPatt wrote: post #102
jonplackett wrote: post #85

Earlier quoted context omitted.

Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.

Except it says in the small print to expect invalid JSON occasionally, so you have to write your error handling code either way

Re: Native JSON Output from GPT-4

#105

Post by yonom »

EGreg wrote: post #95
yonom wrote: post #55

Earlier quoted context omitted.

I would recommend creating a simplified JSON schema for the slides (say, presentation is an array of slides, each slide has a title, body, optional image, optional diagram, each diagram is one of pie, table, ... Then use a library to generate the pptx file from the content generated.

Library? What library? It seems to me that a Transformer should excel at Transforming, say, text into pptx or pdf or HTML with CSS etc. Why don't they train it on that? So I don't have to sit there with manually written libraries. It can easily transform HTML to XML or text bullet points so why not the other formats?

I don't think the name "Transformer" is meant in the sense of "transforming between file formats".

My intuition is that LLMs tend to be good at things human brains are good at (e.g. reasoning), and bad at things human brains are bad at (e.g. math, writing pptx binary files from scratch, ...).

Eventually, we might get LLMs that can open PowerPoint and quickly design the whole presentation using a virtual mouse and keyboard but we're not there yet.

Re: Native JSON Output from GPT-4

#106

Post by ignite »

jonplackett wrote: post #65
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…

> You just append ‘reply in json with this format’ and it does a really good job.

It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.

Re: Native JSON Output from GPT-4

#107

Post by Spivak »

pluijzer wrote: post #46
emilsedgh wrote: post #36

Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful. Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema. One problem that is not…

ChatGPT can be usefully for many things, but you should really, not use it if you want to retrieve factual data. This might partly be resolved by querying the internet like bing does but purely on the language model side these hallucinations are just an unavoidable part of it.

Yep, it's always always write code / query / function / whatever you need that you would parse and retrieve the data from an external system.

Re: Native JSON Output from GPT-4

#108

Post by TeMPOraL »

moneywoes wrote: post #97
swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

Wow your brand is huge. Crazy growth. i wonder how much these subtle mentions on forums help

They're the only one commenter on HN I noticed keeps writing "smol" instead of "small", and is associated with projects with "smol" in their name. Surely I'm not the only one who missed it being a meme around 2015 or sth., and finds this word/use jarring - and therefore very attention-grabbing? Wonder how much that helps with marketing.

This is meant with no negative intentions. It's just that 'swyx was, in my mind, "that HN-er that does AI and keeps saying 'smol'" for far longer than I was aware of latent.space articles/podcasts.

Re: Native JSON Output from GPT-4

#109

Post by EGreg »

yonom wrote: post #105
EGreg wrote: post #95

Earlier quoted context omitted.

Library? What library? It seems to me that a Transformer should excel at Transforming, say, text into pptx or pdf or HTML with CSS etc. Why don't they train it on that? So I don't have to sit there with manually written libraries. It can easily transform HTML to XML or text bullet points so why not the other formats?

I don't think the name "Transformer" is meant in the sense of "transforming between file formats". My intuition is that LLMs tend to be good at things human brains are good at (e.g. reasoning), and bad at things human brains are bad at (e.g. math, writing pptx binary files from scratch, ...). Eventually, we might get LLMs that can open PowerPoint and quickly design the whole presentation using a virtual mouse and key…

It’s just XML They can produce HTML and transform python into php etc.

So why not? It’s easy for them no?

Re: Native JSON Output from GPT-4

#110

Post by davepeck »

SamPatt wrote: post #102
jonplackett wrote: post #85

Earlier quoted context omitted.

Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

99% of the time is still super frustrating when it fails, if you're using it in a consumer facing app. You have to clean up the output to avoid getting an error. If it goes from 99% to 100% JSON that is a big deal for me, much simpler.

Yup. Is there a good/forgiving "drunken JSON parser" library that people like to use? Feels like it would be a useful (and separable) piece?
Post reply on HN