41–50 of 76 posts
Here's 40 lines of python code that I've found to be unreasonably effective at accomplishing something similar: https://github.com/jiggy-ai/pydantic-chatcompletion/blob/mas...
I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…
> Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether it is true or false. Afterwards, use a small embedding model such as sbert to extract true/false from the sentence. We've found that GPT is able to reason better in this case, and it is much more robust. Have you tried just getting it to do both? It reasons far better given some space to think, so I often have it explain thi…
-
column name: salary_max
format: number
example: 150,000
description: Salary Maximum
-
column name: keywords
format: string
example: engineer, python, docker, remote
description: Relevant Keywords (Comma separated keywords used for filtering and matching jobs to candidates)Earlier quoted context omitted.
> Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether it is true or false. Afterwards, use a small embedding model such as sbert to extract true/false from the sentence. We've found that GPT is able to reason better in this case, and it is much more robust. Have you tried just getting it to do both? It reasons far better given some space to think, so I often have it explain thi…
This seems like a better approach. Introducing another unrelated model seems like it would just add an extra point of failure to watch out for.
I'm having success with simple YAML schema. One thing that's very helpful for the prompt is to include "description" and "example": - column name: salary_max format: number example: 150,000 description: Salary Maximum - column name: keywords format: string example: engineer, python, docker, remote description: Relevant Keywords (Comma separated keywords used for filtering and matching jobs to candidates)
I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…
I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…
Maybe not the best thread to ask, but - is there an ELI5 explanation of what exactly CUE is and what is it for? I've landed on that website several times in the last two years, and I could never make heads or tails of it. That the name is just a mix of random, unrelated verbs doesn't help.
I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…
Maybe not the best thread to ask, but - is there an ELI5 explanation of what exactly CUE is and what is it for? I've landed on that website several times in the last two years, and I could never make heads or tails of it. That the name is just a mix of random, unrelated verbs doesn't help.
Also, this: https://bitfieldconsulting.com/golang/cuelang-exciting
I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes it'll triple-quote the JSON string as if it was Python.
My approach to solve these problems was via prompt engineering - using the system message part of the API call. Asking it to "return valid json, do not wrap it in text, do not preface it with text, do not include follow-up explanations, make sure it's valid json, do not include comments" - seems to work 99% of the time. For the remainder, a try-and-catch block with some fallback code that "extracts" json (via dumb REs) from whatever text was returned. Hasn't failed yet.
It's fascinating to watch the new paradigm arrive, and people using old habits to deal with it. This entire project is kind of pointless, you can just ask GPT to return the right kind of thing.
I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…
One thing we have seen is that you need to adjust your prompts when OpenAI updates their model. Given they only support their dated models for so long, it seems increasingly difficult to make the case to build on top of LLMs you cannot control the life cycle for.