It's a shame that the screencast has no sound. I was curious about what it would sound like. I could try it myself via the netlify app but I don't feel very comfortable sharing my API key somewhere...
Show HN: I made a self-hosted ChatGPT UI
21–30 of 54 posts
Re: Show HN: I made a self-hosted ChatGPT UI
#22Earlier quoted context omitted.
I honestly am not that familiar with this space. How realistic is it that someone could self-host a ChatGPT instance? Assuming the model was available, how big are the models and what kind of hardware is necessary to run the instance?
OpenAI hasn't published any information about the size or hardware requirements for running ChatGPT. Reading between the lines, the default ChatGPT Turbo model seems to be significantly smaller than GPT-3 (it's a distilled model), but probably still heavier than the Alpaca and Llama 7B models people are running (very slowly) on their single GPU computers this week. You'd probably need multiple A100s to get comparable…
Does the llama code that dropped leverage the GPU at all? On an M1 it appears to just run on as many CPU cores as you want to throw at it. The 65B heats up 8 cores real nicely, and it's slow, but I imagine it would be a lot faster on the GPU.
Re: Show HN: I made a self-hosted ChatGPT UI
#23Earlier quoted context omitted.
And when you want to create/edit/delete/import custom prompts? AI is a commodity now and a great UX drives adoption.
then you download 1xdeveloper's extension :) though i just copy and paste the bookmarklet and change the prompt
Hehe, I'll put it on github when it reaches 1.0.
Re: Show HN: I made a self-hosted ChatGPT UI
#24It's a shame that the screencast has no sound. I was curious about what it would sound like. I could try it myself via the netlify app but I don't feel very comfortable sharing my API key somewhere...
I posted a screencast on Reddit earlier in the development process with audio demonstrating the text-to-speech feature. The UI has changed a bit since then, but you can hear what the voices sound like:
https://old.reddit.com/r/OpenAI/comments/11k19en/i_made_an_a...
Re: Show HN: I made a self-hosted ChatGPT UI
#25Earlier quoted context omitted.
OpenAI hasn't published any information about the size or hardware requirements for running ChatGPT. Reading between the lines, the default ChatGPT Turbo model seems to be significantly smaller than GPT-3 (it's a distilled model), but probably still heavier than the Alpaca and Llama 7B models people are running (very slowly) on their single GPU computers this week. You'd probably need multiple A100s to get comparable…
Does the llama code that dropped leverage the GPU at all? On an M1 it appears to just run on as many CPU cores as you want to throw at it. The 65B heats up 8 cores real nicely, and it's slow, but I imagine it would be a lot faster on the GPU.
I've seen people saying that limiting it to 4 cores out of the 8 total can actually lead to improved performance. Have you seen that?
Re: Show HN: I made a self-hosted ChatGPT UI
#26Would be cool if they add support for llama.cpp
You really want it integrated with an OpenAI API clone rather than directly integrated. Otherwise, interoperability will suffer greatly as new and improved models are released.
Re: Show HN: I made a self-hosted ChatGPT UI
#27This would be really useful if the API key could be stored in the config file
Re: Show HN: I made a self-hosted ChatGPT UI
#28Earlier quoted context omitted.
Does the llama code that dropped leverage the GPU at all? On an M1 it appears to just run on as many CPU cores as you want to throw at it. The 65B heats up 8 cores real nicely, and it's slow, but I imagine it would be a lot faster on the GPU.
I've seen people saying that limiting it to 4 cores out of the 8 total can actually lead to improved performance. Have you seen that?
8 starts and runs a bit faster for me if plugged in and before the fan kicks on and the CPU starts throttling. Once that happens it's probably better to stick with 4.
Re: Show HN: I made a self-hosted ChatGPT UI
#29I like it. The chat.openai.com frontend is very slow and frequently breaks, so I would consider using this.
Have you considered adding different tts providers? It doesn't get better than elevenlabs right now, but they are also much more expensive than for example the azure neural voices.
Re: Show HN: I made a self-hosted ChatGPT UI
#30> Chat with GPT is an open-source, unofficial ChatGPT app with extra features and more ways to customize your experience. It connects ChatGPT with ElevenLabs to give ChatGPT a realistic human voice. Looks like only GUI aspects of the UI are self-hosted, but that the text and speech aspects of the UI (and the bulk of the computation and IP) are provided by two SaaS services. Self-hosted (and some degree of open) ML mo…
It's somewhat ambiguous language - "self-hosted ChatGPT UI" could lead many to believe it's completely self-hosted.
However, sophisticated readers familiar with ChatGPT will know the model and weights haven't been released and absent a leak/hack/release by OpenAI a completely self-hosted ChatGPT solution is impossible. Eventually we'll almost certainly see a "Completely self-hosted ChatGPT equivalent" (similar to Dall-E vs Stable Diffusion) but that's another thread for another time.
Based on my native speaker parsing of English "Self-hosted ChatGPT UI" is accurate and I'm not sure how else I would write it to disambiguate between a self-hosted UI and a completely self-hosted ChatGPT with a UI.