[dead]
Native JSON Output from GPT-4
171–180 of 258 posts
Re: Native JSON Output from GPT-4
#172I've used GCP Vertex AI for a specific task and the prompt was to generate a JSON response with keys specified and it does generate the result as JSON with said keys.
Re: Native JSON Output from GPT-4
#173Earlier quoted context omitted.
What's the implication of this new change for Microsoft Guidance, LMQL, Langchain, etc.? It looks like much of their functionality (controlling model output) just became obsolete. Am I missing something?
Langchain added support for `function_call` args yesterday: * https://github.com/hwchase17/langchain/pull/6099/files * https://github.com/hwchase17/langchain/issues/6104 IMHO, this should make Langchain much easier and less chaotic to use.
It's only been added to the OpenAI interface. Function calling is really useful when used with agents. To include that to agents would require some redesign as the tool instructions should be removed from the prompt templates in favor of function definitions in the API request. The response parsing code would also be affected.
I just hope they won't come up with yet another agent type.
Re: Native JSON Output from GPT-4
#174Earlier quoted context omitted.
> You just append ‘reply in json with this format’ and it does a really good job. It does an ok job. Except when it doesn't. Definitely misses a lot of the time, sometimes on prompts that succeeded on previous runs.
It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.
Are you saying that it return only JSON before? I'm with the other commenters it was wildly variable and always at least said "Here is your response" which doesn't parse well.
Re: Native JSON Output from GPT-4
#175i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…
Do you people always have to overhype this shit?
Re: Native JSON Output from GPT-4
#176Earlier quoted context omitted.
It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been u…
I could not get GPT-4 to reliably not give some sort of text response, even if was just a simple "Sure" followed by the JSON.
Pass in an agent message with "Sure here is the answer in json format:" after the user message. Gpt will think it has already done the preamble and the rest of the message will start right with the json.
Re: Native JSON Output from GPT-4
#177Earlier quoted context omitted.
It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.
> It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time. Danger! There be assumptions!! gpt-? is a moving target and in rapid development. What it does Tuesday, which it did not do on Monday, it may well not do on W…
ChatGPT moves fast. The API version doesn’t seem to change except with the model and documented API changes.
Re: Native JSON Output from GPT-4
#178Earlier quoted context omitted.
It literally does it everytime perfectly. I remember I put together an entire system that would validate the JSON against a zod schema and use reflection to fix it and it literally never gets triggered because GPT3.5-turbo always does it right the first time.
Are you saying that it return only JSON before? I'm with the other commenters it was wildly variable and always at least said "Here is your response" which doesn't parse well.
If you want a parsable response, have it wrap that with ```. Include an example request/response in your history. Treat any message you can’t parse as an error message.
This works well because it has a place to put any “keep in mind” noise. You can actually include that in your example.
Re: Native JSON Output from GPT-4
#179I thought GPT-4 was doing a pretty good job at outputting JSON (for some of the toy problems I've given it like some of my gardening projects.) Interesting to see this hit the very top of HN
Re: Native JSON Output from GPT-4
#180After reading the docs for the new ChatGPT function calling yesterday, it's structured and/or typed data for GPT input or output that's the key feature of these new models. The ReAct flow of tool selection that it provides is secondary. As this post notes, you don't even need to the full flow of passing a function result back to the model: getting structured data from ChatGPT in itself has a lot of fun and practical…
They have something closer to a simple Hello World example here:
https://platform.openai.com/docs/guides/gpt/function-calling
That example needs a bit of work I think. In Step 3, they're not really using the returned function_name; they're just assuming it's the only function that's been defined, which I guess is equivalent for this simple example with just one function but less instructive. In Step 4, I believe they should also have sent the function definition block again a second time since model calls in the API are memory-less and independent. They didn't, although the model appears to guess what's needed anyway in this case.