51–60 of 76 posts
I Found that LLMs are pretty good with TOML. The multiline strings are also a real bonus. One thing I thought that was interesting is that sometimes the LLM will mistake the triple quoutes for the backticks so it will output something like this ``` [TOML] key=""" value """ """
> Generate some TOML for me, surround the output with a pair of triple equal signs (===).
Typed ChatGPT. Someone had to come along and spoil the fun. Next it'll be ChatGPT/XML.
Is there a reason we won’t have LLMs that only speak in JSON? These JSON hacks are clever and cool but feel like they’ll be obsolete in 6 weeks.
Why do you think they will be obsolete in 6 weeks?
You can use simpler types. Instead of : messages=[ GPTMessage( role=GPTMessageRole.SYSTEM, content=SYSTEM_PROMPT, ), GPTMessage( role=GPTMessageRole.USER, content="Text: I love this product. It's the best thing ever!", ) ] Try: messages=( ("system", SYSTEM_PROMPT), ("user", "Text: I love this product. It's the best thing ever!") ) Or: messages=( SystemMsg(SYSTEM_PROMPT), UserMsg("Text: I love this product. It's the b…
TYPES: [string, int, bool, float, uuid, datetime, email, url]
RELATIONS: [belongs-to, has-one, has-many, many-to-many]
SCHEMA: """
{
"Datamodel": {
"Name": "",
"Models": {
"": {
"": "",
"": "",
"$relations": {
"": {
"name": "",
"type": "",
"model": ""
},
"": {
...
}
}
},
"": {
...
}
}
}
}
"""
I built zod-chatgpt https://github.com/olup/zod-chatgpt a couple of weeks ago to serve same purpose in typescript
I've been interfacing with GPT programmatically for a little while now, leveraging it's "soft and fuzzy" interface to produce hard / machine-readable results. JSON was the format that felt best-suited for the job. I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes…
JSON is admittedly way easier to work with up front, but markdown seems to be a more scaleable choice.
Of course, this is all very much an opinion and highly anecdotal at the moment.
I've been interfacing with GPT programmatically for a little while now, leveraging it's "soft and fuzzy" interface to produce hard / machine-readable results. JSON was the format that felt best-suited for the job. I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes…
Also note that the output still depends on a random sampling of the next token according to the distribution that the net gives you - so there is a lot of genuine randomness in the model's behaviour. And because each sampled token influences the rest of the response, this randomness will become stronger the longer the response is.
So if you already know you're only interested in a particular subset of tokens, it makes sense to me to clamp the distribution to only those tokens and keep the model from getting onto the "wrong path" in the first place.
Also, pragmatically, if you can get the model to restrict itself to JSON without telling it in the prompt, you're saving that part if the context window for better uses.
Is there a reason we won’t have LLMs that only speak in JSON? These JSON hacks are clever and cool but feel like they’ll be obsolete in 6 weeks.
Technically speaking it's pretty to force the model into an valid JSON-schema if you have access to the inference autoregressive loop and the logit activations. You can either force known areas of the template to a prefixed template, or fill in the basic JSON wrapper and force the model to choose arbitrary keys and values.
Imagine it might come down to how much of their usage is on generating text vs generating structured prediction payloads.
I've been interfacing with GPT programmatically for a little while now, leveraging it's "soft and fuzzy" interface to produce hard / machine-readable results. JSON was the format that felt best-suited for the job. I see a ton of code in this project, and I don't know what most of it does. As far as GPT troubles with JSON, I'll add a couple: sometimes it likes to throw comments in there as if it was JS. And sometimes…
But the problem is the 99%, no ?
Why do we need a library for this?