Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything ChatGPT – under the hood of the ChatGPT web app

github.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#21

Post by black3r »

1xdevloper wrote: post #18
chatmasta wrote: post #5

This doesn't actually cover the most important part, which is where the response to a chat shows up. Has anyone figured this out? I only looked briefly, but AFAICT, it doesn't show up in the response data in Chrome or Firefox DevTools. I figured it was maybe using WebSockets or Server Sent Events, which apparently don't show up in DevTools in some cases (?). I think this is a Chrome bug [0] [1] and also a Firefox bug…

Probably because they're using a polyfill[0][1] to handle the event stream. [0] https://stackoverflow.com/questions/55201372/event-stream-da... [1] https://github.com/EventSource/eventsource/issues/94

And that's because the native EventStream API only supports GET requests, so if you want to use a POST request (which make more sense in this context) you have to use a custom implementation...

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#22

Post by qqqturing2 »

chatmasta wrote: post #5

This doesn't actually cover the most important part, which is where the response to a chat shows up. Has anyone figured this out? I only looked briefly, but AFAICT, it doesn't show up in the response data in Chrome or Firefox DevTools. I figured it was maybe using WebSockets or Server Sent Events, which apparently don't show up in DevTools in some cases (?). I think this is a Chrome bug [0] [1] and also a Firefox bug…

they are using Server Side Events (SSE) to stream the text.

see API docs: https://platform.openai.com/docs/api-reference/chat/create

> If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. See the OpenAI Cookbook for example code.

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#23

Post by czx4f4bd »

qqqturing2 wrote: post #22
chatmasta wrote: post #5

This doesn't actually cover the most important part, which is where the response to a chat shows up. Has anyone figured this out? I only looked briefly, but AFAICT, it doesn't show up in the response data in Chrome or Firefox DevTools. I figured it was maybe using WebSockets or Server Sent Events, which apparently don't show up in DevTools in some cases (?). I think this is a Chrome bug [0] [1] and also a Firefox bug…

they are using Server Side Events (SSE) to stream the text. see API docs: https://platform.openai.com/docs/api-reference/chat/create > If set, partial message deltas will be sent, like in ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message. See the OpenAI Cookbook for example code.

Interesting. Apparently server-sent events show up in devtools under the network tab as multiple responses sent for the same request.

https://firefox-source-docs.mozilla.org/devtools-user/networ...

https://stackoverflow.com/a/46075339/6417784

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#24

Post by tercmd »

chatmasta wrote: post #5

This doesn't actually cover the most important part, which is where the response to a chat shows up. Has anyone figured this out? I only looked briefly, but AFAICT, it doesn't show up in the response data in Chrome or Firefox DevTools. I figured it was maybe using WebSockets or Server Sent Events, which apparently don't show up in DevTools in some cases (?). I think this is a Chrome bug [0] [1] and also a Firefox bug…

(Creator of the repo here) That is somewhat present in the `/backend-api/moderations` endpoint where the app sends your question + ChatGPT's response to (first time just your question, second time both)

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#25

Post by tercmd »

tylerekahn wrote: post #3

The web app also has built in support for LaTeX rendering (it uses KaTeX). See this forum post: https://community.openai.com/t/new-feature-properly-shown-ma... You can get ChatGPT to output conforming LaTeX expressions using the following prompt: Could you please use the specific LaTeX math mode delimiters for your response? LaTex math mode specific delimiters as following 1. inline math mode : `\(` and `\)` 2. displ…

In some of the ChatGPT client-side JS (visible in the Debugger tab of DevTools), I could see references like `\triangle`. However, asking ChatGPT to print either as it is just prints the text itself.

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#26

Post by tercmd »

quenix wrote: post #2

Interesting—so it seems there is an 'exploit' which lets non-Plus subscribers hit an internal API endpoint to send themselves a special Plus login URL (for times of peak demand): https://github.com/terminalcommandnewsletter/everything-chat...

I've not tested it further, but I wondered what happened when I saw a "Plus subscriber login link" input field - resulted in that. In fact, I wonder if one could automate the request to the API endpoint (all headers, authorization and everything) when the browser sees a "ChatGPT down" message. (Of course, you have to click the link in the email, but if you grant the tool access to your email to automate that, that probably isn't the best thing for "principle of least privilege")

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#28

Post by tylerekahn »

tercmd wrote: post #25
tylerekahn wrote: post #3

The web app also has built in support for LaTeX rendering (it uses KaTeX). See this forum post: https://community.openai.com/t/new-feature-properly-shown-ma... You can get ChatGPT to output conforming LaTeX expressions using the following prompt: Could you please use the specific LaTeX math mode delimiters for your response? LaTex math mode specific delimiters as following 1. inline math mode : `\(` and `\)` 2. displ…

In some of the ChatGPT client-side JS (visible in the Debugger tab of DevTools), I could see references like `\triangle`. However, asking ChatGPT to print either as it is just prints the text itself.

This prompt works best with GPT 4. GPT 3.5 gives inconsistent results. There might be a way of improving the prompt for 3.5

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#29

Post by tercmd »

tercmd wrote: post #25
tylerekahn wrote: post #3

The web app also has built in support for LaTeX rendering (it uses KaTeX). See this forum post: https://community.openai.com/t/new-feature-properly-shown-ma... You can get ChatGPT to output conforming LaTeX expressions using the following prompt: Could you please use the specific LaTeX math mode delimiters for your response? LaTex math mode specific delimiters as following 1. inline math mode : `\(` and `\)` 2. displ…

In some of the ChatGPT client-side JS (visible in the Debugger tab of DevTools), I could see references like `\triangle`. However, asking ChatGPT to print either as it is just prints the text itself.

Turns out, this actually does work.

Re: Everything ChatGPT – under the hood of the ChatGPT web app

#30

Post by tercmd »

1xdevloper wrote: post #17

The website also calls the moderation API from the client side with the response returned from its conversation API. So if you simply block the request to the moderation endpoint in dev tools, do they still have additional built-in monitoring?

If you block the `/backend-api/moderations` endpoint, that (as expected) doesn't block the AI from not giving you an answer.
Post reply on HN