Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Native JSON Output from GPT-4

yonom.substack.com

Post reply on HN
Print view View original

121–130 of 258 posts

Re: Native JSON Output from GPT-4

#121

Post by 8organicbits »

jonplackett wrote: post #85
cwxm wrote: post #74

Earlier quoted context omitted.

even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.

Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

Wouldn't you use traditional software to validate the JSON, then ask chatgpt to try again if it wasn't right?

Re: Native JSON Output from GPT-4

#122

Post by Tostino »

londons_explore wrote: post #94
Tostino wrote: post #64

Earlier quoted context omitted.

Not GP, but it's still the latter...i've been (im)patiently waiting. From their blog post the other day: With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!

If you put contact info in your HN profile - especially an email address that matches one you use to login to openai, someone will probably give you access... Anyone with access can share it with any other user via the 'invite to organisation' feature. Obviously that allows the invited person do requests billed to the inviter, but since most experiments are only a few cents that doesn't really matter much in practice…

Good to know, but I've racked up a decent bill for just my GPT 3.5 use. I can get by with experiments using my ChatGPT Plus subscription, but I really need my own API access to start using it for anything serious.

Re: Native JSON Output from GPT-4

#123

Post by neuronexmachina »

behnamoh wrote: post #44
minimaxir wrote: post #3

After reading the docs for the new ChatGPT function calling yesterday, it's structured and/or typed data for GPT input or output that's the key feature of these new models. The ReAct flow of tool selection that it provides is secondary. As this post notes, you don't even need to the full flow of passing a function result back to the model: getting structured data from ChatGPT in itself has a lot of fun and practical…

What's the implication of this new change for Microsoft Guidance, LMQL, Langchain, etc.? It looks like much of their functionality (controlling model output) just became obsolete. Am I missing something?

Langchain added support for `function_call` args yesterday:

* https://github.com/hwchase17/langchain/pull/6099/files

* https://github.com/hwchase17/langchain/issues/6104

IMHO, this should make Langchain much easier and less chaotic to use.

Re: Native JSON Output from GPT-4

#124

Post by Sai_ »

m3kw9 wrote: post #119
Sai_ wrote: post #115

Earlier quoted context omitted.

> fuzzy logic Yes and no. While the choice of which function to call is dependent on an llm, ultimately, you control the function itself whose output is deterministic. Even today, given an api, people can choose to call or not call based on some factor. We don’t call this fuzzy logic. E.g., people can decide to sell or buy stock through an api based on some internal calculations - doesn’t make the system “fuzzy”.

If you feed that result into another io box you may or may not know if that is the correct answer, which may need some sort of error detection. I think this is going to be majority of the use cases

Hm, I see what you mean. Afaict, only the decision to call or not call a function is up to the model (fuzzy). Once it decides to call the function, it generates mostly correct JSON based on your schema and returns that to you as is (not very fuzzy).

It’ll be interesting to test APIs which accept user inputs. Depending on how ChatGPT populates the JSON, the API could be required to understand/interpret/respond to lots of variability in inputs.

Re: Native JSON Output from GPT-4

#125

Post by bradly »

rytill wrote: post #100
bradly wrote: post #82

Earlier quoted context omitted.

I could not get GPT-4 to reliably not give some sort of text response, even if was just a simple "Sure" followed by the JSON.

Did you try using the API and providing a very clear system message followed by several examples that were pure JSON?

Yep. I even gave it a JSON schema file to use. It just wouldn't stop added extra verbage.

Re: Native JSON Output from GPT-4

#126

Post by runeb »

The way openai implemented this is really clever, beyond how neat the plugin architecture is, as it lets them peek one layer inside your internal API surface and can infer what you intend to do with the LLM output. Collecting some good data here.

Re: Native JSON Output from GPT-4

#127

Post by m3kw9 »

Sai_ wrote: post #124
m3kw9 wrote: post #119

Earlier quoted context omitted.

If you feed that result into another io box you may or may not know if that is the correct answer, which may need some sort of error detection. I think this is going to be majority of the use cases

Hm, I see what you mean. Afaict, only the decision to call or not call a function is up to the model (fuzzy). Once it decides to call the function, it generates mostly correct JSON based on your schema and returns that to you as is (not very fuzzy). It’ll be interesting to test APIs which accept user inputs. Depending on how ChatGPT populates the JSON, the API could be required to understand/interpret/respond to lots…

Yeah I’ve tested, you should use the curl example they gave as you can test instantly pasting it into your terminal. The description of the functions is prompt engineering in addition to the original system prompt, need to test the dependency more, it’s so new.

Re: Native JSON Output from GPT-4

#128

Post by abhpro »

edwin wrote: post #41

For those who want to test out the LLM as API idea, we are building a turnkey prompt to API product. Here's Simon's recipe maker deployed in a minute: https://preview.promptjoy.com/apis/1AgCy9 . Public preview to make and test your own API: https://preview.promptjoy.com

This is really cool, I had a similar idea but didn't build it. I was also thinking a user could take these different prompts (I called them tasks) that anyone could create, and then connect them together like a node graph or visual programming interface, with some Chat-GPT middleware that resolves the outputs to inputs.

Re: Native JSON Output from GPT-4

#129

Post by freezed88 »

swyx wrote: post #19

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday ( https://www.latent.space/p/function-agents ) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functi…

100%, if the API itself can choose to call a function or an LLM, then it's way easier to build any agent loop without extensive prompt engineering + worrying about errors.

Tweeted about it here as well: https://twitter.com/jerryjliu0/status/1668994580396621827?s=...

Re: Native JSON Output from GPT-4

#130

Post by NicoJuicy »

bradly wrote: post #125
rytill wrote: post #100

Earlier quoted context omitted.

Did you try using the API and providing a very clear system message followed by several examples that were pure JSON?

Yep. I even gave it a JSON schema file to use. It just wouldn't stop added extra verbage.

Say it's a json API and may only reply with valid json without explanation.
Post reply on HN