Show HN: GPT-JSON – Structured and typehinted GPT responses in Python • Hacker News Board

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#31

Post by reissbaker » Fri, May 05, 2023, 9:12 AM UTC

Hey, this is really neat! I've taken a very similar approach in TypeScript. +1 to the sibling comment that recommended parsing with json5 (but don't tell the AI you're doing that, it's a waste of context space and it might get more confused anyway).

I've had luck doing chain-of-thought prompting in the JSON payload as you've described, too. Cheers, really validating to see someone taking a similar approach.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#32

Post by TeMPOraL » Fri, May 05, 2023, 9:23 AM UTC

teleforce wrote: post #23

I asked ChatGPT (GPT-4) about the idea of integrating GPT-JSON with CUE language ( https://cuelang.org/ ) for potential improvements and the answers are quite impressive: "Yes, integrating the CUE language with GPT-JSON can potentially lead to improvements in its capability. CUE, short for Configuration, Unification, and Environment, is an open-source data constraint language designed to simplify the definition, gene…

Maybe not the best thread to ask, but - is there an ELI5 explanation of what exactly CUE is and what is it for? I've landed on that website several times in the last two years, and I could never make heads or tails of it. That the name is just a mix of random, unrelated verbs doesn't help.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#33

Post by BiteCode_dev » Fri, May 05, 2023, 9:27 AM UTC

rain1 wrote: post #30

BiteCode_dev wrote: post #24

You can use simpler types. Instead of : messages=[ GPTMessage( role=GPTMessageRole.SYSTEM, content=SYSTEM_PROMPT, ), GPTMessage( role=GPTMessageRole.USER, content="Text: I love this product. It's the best thing ever!", ) ] Try: messages=( ("system", SYSTEM_PROMPT), ("user", "Text: I love this product. It's the best thing ever!") ) Or: messages=( SystemMsg(SYSTEM_PROMPT), UserMsg("Text: I love this product. It's the b…

python newbie here, why did messages change from [ to (

Doesn't matter much in this particular case, one is a tuple, the other is a list, both can be read with a for loop.

The important part is that the way messages types are defined is very heavy, and you can get them to be lighter without sacrificing type safety.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#34

Post by robflaherty » Fri, May 05, 2023, 9:38 AM UTC

Is there a reason we won’t have LLMs that only speak in JSON? These JSON hacks are clever and cool but feel like they’ll be obsolete in 6 weeks.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#35

Post by fswd » Fri, May 05, 2023, 9:42 AM UTC

Ha, I just spent two hours hacking something up similar and I eventually just sent a system role with zod schema in the content

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#36

Post by valyagolev » Fri, May 05, 2023, 9:57 AM UTC

the dirtyjson library has been doing this work for me quite well

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#37

Post by TeeWEE » Fri, May 05, 2023, 10:59 AM UTC

wskish wrote: post #7

Here's 40 lines of python code that I've found to be unreasonably effective at accomplishing something similar: https://github.com/jiggy-ai/pydantic-chatcompletion/blob/mas...

I dont see how these infinite loops are a good idea... You never sure if you actually getting a good result ?

What is the failure rate?

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#38

Post by ZephyrBlu » Fri, May 05, 2023, 11:14 AM UTC

crazyedgar wrote: post #3

I like the idea, but I think a library that focuses on producing requests and parsing responses according to schema is better. Sending requests to the server is orthogonal to the purpose. What we've found useful in practice in dealing with similar problems: - Use json5 instead of json when parsing. It allows trailing commas. - Don't let it respond in true/false. Instead, ask it for a short sentence explaining whether…

Didn't know about json5, so I had to deal with trailing commas in another way. I found that providing an example of an array without trailing commas was enough for GPT to pick up on it.

The tips on booleans and numerics are interesting! Will keep them in mind if I ever need to do that. I've definitely experienced a few quirks like that (E.g. ChatGPT 'helpfully' responding with "Here's your JSON" instead of just giving me JSON).

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#39

Post by ZephyrBlu » Fri, May 05, 2023, 11:16 AM UTC

TeeWEE wrote: post #37

wskish wrote: post #7

Here's 40 lines of python code that I've found to be unreasonably effective at accomplishing something similar: https://github.com/jiggy-ai/pydantic-chatcompletion/blob/mas...

I dont see how these infinite loops are a good idea... You never sure if you actually getting a good result ? What is the failure rate?

Um, what? This is a standard retry loop. It's just generally good practice.

Re: Show HN: GPT-JSON – Structured and typehinted GPT responses in Python

#40

Post by Loeffelmann » Fri, May 05, 2023, 11:44 AM UTC

I Found that LLMs are pretty good with TOML. The multiline strings are also a real bonus. One thing I thought that was interesting is that sometimes the LLM will mistake the triple quoutes for the backticks so it will output something like this

  ```
  [TOML]
  key="""
  value
  """
  """