For those who want to test out the LLM as API idea, we are building a turnkey prompt to API product. Here's Simon's recipe maker deployed in a minute: https://preview.promptjoy.com/apis/1AgCy9 . Public preview to make and test your own API: https://preview.promptjoy.com
Native JSON Output from GPT-4
41–50 of 258 posts
Re: Native JSON Output from GPT-4
#42Nice to have an endpoint which takes care of this. I've been doing this manually, it's a fairly simple process:
* Add "Output your response in json format, with the fields 'x', which indicates 'x_explanation', 'z', which indicates 'z_explanation' (...)" etc. GPT-4 does this fairly reliably.
* Validate the response, repeat if malformed.
* Bam, you've got a json.
I wonder if they've implemented this endpoint with validation and carefully crafted prompts on the base model, or if this is specifically fine-tuned.
Re: Native JSON Output from GPT-4
#43Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful. Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema. One problem that is not…
I would never rely on an LLM as a source of such information, just as I wouldn't trust the general knowledge of a human being used as a database. Does your workflow include a step for information search? With the new json features, it should be easy to instruct it to perform a search or directly feed it the right pages to parse.
Re: Native JSON Output from GPT-4
#44After reading the docs for the new ChatGPT function calling yesterday, it's structured and/or typed data for GPT input or output that's the key feature of these new models. The ReAct flow of tool selection that it provides is secondary. As this post notes, you don't even need to the full flow of passing a function result back to the model: getting structured data from ChatGPT in itself has a lot of fun and practical…
What's the implication of this new change for Microsoft Guidance, LMQL, Langchain, etc.? It looks like much of their functionality (controlling model output) just became obsolete. Am I missing something?
Re: Native JSON Output from GPT-4
#45Nice to have an endpoint which takes care of this. I've been doing this manually, it's a fairly simple process: * Add "Output your response in json format, with the fields 'x', which indicates 'x_explanation', 'z', which indicates 'z_explanation' (...)" etc. GPT-4 does this fairly reliably. * Validate the response, repeat if malformed. * Bam, you've got a json. I wonder if they've implemented this endpoint with valid…
It appears to be fine-tuning:
"These models have been fine-tuned to both detect when a function needs to be called (depending on the user’s input) and to respond with JSON that adheres to the function signature."
https://openai.com/blog/function-calling-and-other-api-updat...
Re: Native JSON Output from GPT-4
#46Building agents that use advanced API's was not really practical until now. Things like Langchain's Structured Agents worked somewhat reliably, but due to the massive token count it was so slow, the experience was _never_ going to be useful. Due to this, the performance in which our agent processes results has improved 5-6 times and it does actually do a pretty good job of keeping the schema. One problem that is not…
ChatGPT can be usefully for many things, but you should really, not use it if you want to retrieve factual data. This might partly be resolved by querying the internet like bing does but purely on the language model side these hallucinations are just an unavoidable part of it.
Re: Native JSON Output from GPT-4
#47Actually I'm looking to take GPT-4 output and create file formats like keynote presentations, or pptx. Is that currently possible with some tools?
Re: Native JSON Output from GPT-4
#48Glad we didn't get to far into adopting something like Guardrails. This sort of kills it's main value prop for OpenAI. https://shreyar.github.io/guardrails/
Luckily it's for LLMs, not openai
Re: Native JSON Output from GPT-4
#49After reading the docs for the new ChatGPT function calling yesterday, it's structured and/or typed data for GPT input or output that's the key feature of these new models. The ReAct flow of tool selection that it provides is secondary. As this post notes, you don't even need to the full flow of passing a function result back to the model: getting structured data from ChatGPT in itself has a lot of fun and practical…
What's the implication of this new change for Microsoft Guidance, LMQL, Langchain, etc.? It looks like much of their functionality (controlling model output) just became obsolete. Am I missing something?
[deleted]
Re: Native JSON Output from GPT-4
#50It's a shame they couldn't use yaml, instead. I compared them and yaml uses about 20% fewer tokens. However, I can understand accuracy, derived from frequency, being more important than token budget.