Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

github.com

Post reply on HN
Print view View original

1–10 of 76 posts

Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#1

Post by icyfox »

Hey HN, I've been using GPT a lot lately in some side projects around data generation and benchmarking. During the course of prompt tuning I ended up with a pretty complicated request: the value that I was looking for, an explanation, a criticism, etc. JSON was the most natural output format for this but results would often be broken, have wrong types, or contain missing fields.

There's been some positive movement in this space, like with jsonformer (https://github.com/1rgs/jsonformer) the other day. But nothing that was plug and play with GPT.

This library consolidates the separate logic that I built across 5 different projects. It lets you prompt the model for how it should return fields, inject variable prompts, handle common formatting errors, then cast to pydantic when you're done for typehinting and validation in your IDE. If you're able to play around with it, let me know what you think.

Show HN: GPT-JSON – Structured and typehinted GPT responses in Python
github.com

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#3

Post by crazyedgar »

I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose.

What we've found useful in practice in dealing with similar problems:

- Use json5 instead of json when parsing. It allows trailing commas.

- Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether it is true or false. Afterwards, use a small embedding model such as sbert to extract true/false from the sentence. We've found that GPT is able to reason better in this case, and it is much more robust.

- For numerical scores, do a similar thing by asking GPT for a description, then with the small embedding model write a few examples matching your score scale, and for each response use the score of the best matched example. If you let GPT give you scores directly without explanation, 20% of the time it will give you nonsense.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#6

Post by icyfox »

crazyedgar wrote: post #3

I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…

Thanks for the thoughts! I've deployed a few meta models that act like you're describing for second stage predictions, but for fuzzy task definitions have actually seen similar luck with having GPT explicitly explain its rational and then force it to choose a true/false rating. My payloads often end up looking like:

  class Payload:
    reasoning: str = Field(description="Why this value might be true or false),
    answer: bool
Since it's autoregressive I imagine the schema helps to define the universe of what it's supposed to do, then the decoder attention when it's filling the `answer` can look back on the reasoning and weigh the sentiment internally. I imagine the accuracy specifics depend a lot on the end deployment here.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#8

Post by iamflimflam1 »

crazyedgar wrote: post #3

I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…

There’s some really good info along thff we same lines in this course https://learn.deeplearning.ai/chatgpt-prompt-eng

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#9

Post by yawnxyz »

m3kw9 wrote: post #4

Wasn’t there a yesterday proj that asks gpt to output non json but then it turns it into json after? Thus making it less prone to json errors

That one is robust but requires pytorch and huggingface and all kinds of things. I think most of us want something that's reasonable few lines of code and can run in places like serverless and replit.
Post reply on HN