Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

github.com

Post reply on HN
Print view View original

61–70 of 76 posts

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#61

Post by zh217 »

wskish wrote: post #7

Here's 40 lines of python code that I've found to be unreasonably effective at accomplishing something similar: https://github.com/jiggy-ai/pydantic-chatcompletion/blob/mas...

Thanks. Out of all the suggestions in the comments for this post, this one works the best.

And in fact it is only one line, not 40:

    "Please respond ONLY with valid json that conforms to this pydantic json_schema: {model_class.schema_json()}. Do not include additional text other than the object json as we will load this object with json.loads() and pydantic."

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#62

Post by icyfox »

teleforce wrote: post #23

I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…

Pretty good pitch from ChatGPT if you ask me! I haven't used cue outside of test projects before, so a few questions:

1. Is cue's validation a material improvement from something like pydantic or zod, which defines schema as code versus in .cue files? I see their docs argue that this can allow for client-side validation and lighter weight schema files which doesn't seem to totally address the library side of things.

2. Have you used the scripting layer before and do you find it useful in practice? I'm struggling a bit to see how I or GPT would use this in my day-to-day.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#63

Post by zh217 »

crazyedgar wrote: post #3

I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…

Another alternative JSON parser is the YAML parser. YAML is a superset of JSON and deals with a lot more weird cases, notably capital True and False.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#64

Post by icyfox »

mmaia wrote: post #43

I'm having success with simple YAML schema. One thing that's very helpful for the prompt is to include "description" and "example": - column name: salary_max format: number example: 150,000 description: Salary Maximum - column name: keywords format: string example: engineer, python, docker, remote description: Relevant Keywords (Comma separated keywords used for filtering and matching jobs to candidates)

Good suggestion mmaia - I'm opening a new issue to keep track of the different output schemas that are being suggested. One thing that originally worried me a bit with yaml was its relative reliance on space-based formatting to drive meaning. GPT generally tokenizes newlines/spaces but a lot of preprepared datasets strip these out, so I preferred the explicitness of a json that's idempotent to spaces. Have you tried using this approach for non-tabular or nested data like lists or dictionaries?

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#65

Post by icyfox »

verdverm wrote: post #60
olabyne wrote: post #56

Earlier quoted context omitted.

But the problem is the 99%, no ?

It works fine 99% of the time by just using a small amount of extra instruction in the actual prompt. The method GP describes works in any language with just the basic building blocks of http requests, regexp, and a json decoder. Why do we need a library for this?

You might not! Depends on what you're looking for. I've been finding this library most helpful in places where I have a lot of GPT calls in pipelines, so having typehinted schema return values / some built in error correction / variable injection / establishing standards for the IO of prompt schema is the most useful. So IMO I see its main use as a good standard set of operations that work pretty well out of the box and that allow you to hack around them with decent flexibility.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#66

Post by alalalal »

AmazingTurtle wrote: post #26

What about few shot learning, e.g. injecting 1 or 2 examples of JSON in the prompt? That should be fine as well

I literally tried that yesterday. You will be introducing bias. It works well until you infer with something very similar to one of the examples you used in the prompt but not exactly the same then it will always return the json you used in your example.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#67

Post by pigcat »

Since others are sharing their prompt-only solutions to get JSON, I'll share what I've been using. Has been working reliably:

"Do not include any explanations, only provide a RFC8259 compliant JSON response following this format without deviation.

{

  "author": "string describing the author full name",

  "year": "number describing the year the book was written",

  "isFiction": "boolean describing if the book is a work of fiction"

  ...
} "

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#68

Post by btbuildem »

xg15 wrote: post #58
btbuildem wrote: post #49

I've been interfacing with GPT programmatically for a little while now, leveraging it's "soft and fuzzy" interface to produce hard / machine-readable results. JSON was the format that felt best-suited for the job. I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes…

Why not both? You can tell it in the prompt what you want and still constrain the output programmatically. Also note that the output still depends on a random sampling of the next token according to the distribution that the net gives you - so there is a lot of genuine randomness in the model's behaviour. And because each sampled token influences the rest of the response, this randomness will become stronger the long…

I agree with others that it would be interesting to see an LLM that outputs JSON natively - but I think it would also be moving in the opposite direction of the general trend. Right now I can ask it for JSON, YAML, or a number of other formats.

To answer "why not both?" -- bottom line, the effort involved. I don't want to deal with yet another library, the bugs in it, and the inevitable -changes-. GPT's capacity for bridging the gap between structured and human languages is an enormous boon. It bridges a gap so large, we most of the time can't span it with our imaginations. I don't need to write code to tell GPT what to do, I can direct it in plain english.

I'm not worried about the size of the context window the same way we're not worried about memory or disk space - there will be more.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#69

Post by verdverm »

icyfox wrote: post #65
verdverm wrote: post #60

Earlier quoted context omitted.

It works fine 99% of the time by just using a small amount of extra instruction in the actual prompt. The method GP describes works in any language with just the basic building blocks of http requests, regexp, and a json decoder. Why do we need a library for this?

You might not! Depends on what you're looking for. I've been finding this library most helpful in places where I have a lot of GPT calls in pipelines, so having typehinted schema return values / some built in error correction / variable injection / establishing standards for the IO of prompt schema is the most useful. So IMO I see its main use as a good standard set of operations that work pretty well out of the box…

yeah, I don't disagree, however this idea is better a function I use within gpt-index or langchain. There is a horse race and we're all making our bets about who's going to win

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#70

Post by mhsred5 »

icyfox wrote: post #62
teleforce wrote: post #23

I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…

Pretty good pitch from ChatGPT if you ask me! I haven't used cue outside of test projects before, so a few questions: 1. Is cue's validation a material improvement from something like pydantic or zod, which defines schema as code versus in .cue files? I see their docs argue that this can allow for client-side validation and lighter weight schema files which doesn't seem to totally address the library side of things.…

CUE's validation is very strong, as long as you're ok with some level of functional programming and immutability.

Despite what GPT said above, its Configure, Unify, Execute. The Execute aspect is powerful but you have to be ok with functional programming and immutability.

Pairs up really nicely with Go.

Post reply on HN