In what way is this a “list of dicts”? It’s an implicitly structured list of dict, str pairs. And it is WEIRD.
yeah it doesn't match the API format anyway, which is a list of dicts.
ChatML: ChatGPT API expects a structured format, called Chat Markup Language
21–30 of 83 posts
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#22(I work at OpenAI.) This document is a preview of the underlying format consumed by ChatGPT models. As an API user, today you use our higher-level API ( https://platform.openai.com/docs/guides/chat ). We'll be opening up direct access to this format in the future, and want to give people visibility into what's going on under the hood in the meanwhile!
Is there a way for us to have more users in the chat? We are working on a group chat implementation for augmenting conversations and I’m curious if ChatML will easily accommodate it.
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#23Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#24(I work at OpenAI.) This document is a preview of the underlying format consumed by ChatGPT models. As an API user, today you use our higher-level API ( https://platform.openai.com/docs/guides/chat ). We'll be opening up direct access to this format in the future, and want to give people visibility into what's going on under the hood in the meanwhile!
Not to sound rude, but how are you guys going to determine differences between user input and say, an input from an external sources like pdf, email, webpage, webapps? Do you have thoughts on it? If I make an application, I will want to link to external systems.
If there isn’t any way to distinguish it, I bet the attack surface is too large. If it is restricted to QA without external interface, then usability is also restricted. Any thoughts about it?
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#25Why do you think Bing Chat is going in a slightly different direction and not using this format exactly? Their prompt - https://old.reddit.com/r/bing/comments/11bd91j/release_of_th... Wouldn't it be better to unify around ChatML, for the sake of all future training data being consistent? I thought it was strange that they used but not the rest of the ChatML syntax. (There is always the possibility that they are using…
I'm VERY curious about those "#inner_monologue" sections...
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#26Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#27Why do you think Bing Chat is going in a slightly different direction and not using this format exactly? Their prompt - https://old.reddit.com/r/bing/comments/11bd91j/release_of_th... Wouldn't it be better to unify around ChatML, for the sake of all future training data being consistent? I thought it was strange that they used but not the rest of the ChatML syntax. (There is always the possibility that they are using…
I'm VERY curious about those "#inner_monologue" sections...
Looks like it's just getting it to "show its work" to improve the output.
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#28(I work at OpenAI.) This document is a preview of the underlying format consumed by ChatGPT models. As an API user, today you use our higher-level API ( https://platform.openai.com/docs/guides/chat ). We'll be opening up direct access to this format in the future, and want to give people visibility into what's going on under the hood in the meanwhile!
There doesn't seem to be any way to protect against prompt injection attacks against [system], since [system] isn't a separate token. I understand this is a preview, but if there's one takeaway from the history of cybersecurity attacks, it's this: please put some thought into how queries are escaped. SQL injection attacks plagued the industry for decades precisely because the initial format didn't think through how t…
Good catch. They call this "ChatML v0", not "v1", so I'd guess they realize that it looks more like an internal implementation kludge, than an exposed interface.
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#29Why do you think Bing Chat is going in a slightly different direction and not using this format exactly? Their prompt - https://old.reddit.com/r/bing/comments/11bd91j/release_of_th... Wouldn't it be better to unify around ChatML, for the sake of all future training data being consistent? I thought it was strange that they used but not the rest of the ChatML syntax. (There is always the possibility that they are using…
Platform lock-in.
Interoperability would not only be more costly in terms of the work to design and agree on a common standard, but it'd also makes it easier to switch to a competitor.
Re: ChatML: ChatGPT API expects a structured format, called Chat Markup Language
#30(I work at OpenAI.) This document is a preview of the underlying format consumed by ChatGPT models. As an API user, today you use our higher-level API ( https://platform.openai.com/docs/guides/chat ). We'll be opening up direct access to this format in the future, and want to give people visibility into what's going on under the hood in the meanwhile!
There doesn't seem to be any way to protect against prompt injection attacks against [system], since [system] isn't a separate token. I understand this is a preview, but if there's one takeaway from the history of cybersecurity attacks, it's this: please put some thought into how queries are escaped. SQL injection attacks plagued the industry for decades precisely because the initial format didn't think through how t…
One detail you may have missed — "system" is only special when it comes right after a special token. So it's not a special token itself, but you cannot inject a valid-looking system message from user text.
In more detail, the current format is:
HEADER BODY
We are actually going to swap over to this shortly:
HEADERBODY
So basically getting rid of the newline separator and replacing with a special token. Shouldn't change anything fundamentally, but does help with some whitespace tokenization-related issues.
BTW, format of HEADER is going to be really interesting, there's all sorts of metadata one might want to add in there — and making sure that its extensible and not injectable will be an ongoing part of the design work!