Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made a self-hosted ChatGPT UI

github.com

Post reply on HN
Print view View original

11–20 of 54 posts

Re: Show HN: I made a self-hosted ChatGPT UI

#11

Post by tottenval »

whartung wrote: post #3
neilv wrote: post #2

> Chat with GPT is an open-source, unofficial ChatGPT app with extra features and more ways to customize your experience. It connects ChatGPT with ElevenLabs to give ChatGPT a realistic human voice. Looks like only GUI aspects of the UI are self-hosted, but that the text and speech aspects of the UI (and the bulk of the computation and IP) are provided by two SaaS services. Self-hosted (and some degree of open) ML mo…

I honestly am not that familiar with this space. How realistic is it that someone could self-host a ChatGPT instance? Assuming the model was available, how big are the models and what kind of hardware is necessary to run the instance?

OpenAI hasn't published any information about the size or hardware requirements for running ChatGPT. Reading between the lines, the default ChatGPT Turbo model seems to be significantly smaller than GPT-3 (it's a distilled model), but probably still heavier than the Alpaca and Llama 7B models people are running (very slowly) on their single GPU computers this week. You'd probably need multiple A100s to get comparable performance to the ChatGPT API.

Re: Show HN: I made a self-hosted ChatGPT UI

#12

Post by TOMDM »

prenoob wrote: post #9

What I think i need is something like this, but in bookmarklet form. I click it, it prompt()s me for the prompt and displays the output in a textarea so i can quickly paste it. Thinking of it it should be possible to put the output straight into the clipboard, right? The use case of course would be email/forum communication. The problem is that you have to make a UI to embed the API key into the code, because pasting…

What I think would be cool is taking automatically from highlighted text in any app, falling back to my clipboard as input, and then outputting to my clipboard.

That way it works in any app automatically. Seamless system wide clipboard read is a big ask though, so ideally you'd want a self hosted model like llama.cpp

Re: Show HN: I made a self-hosted ChatGPT UI

#15

Post by 1xdevloper »

ChatGPT API can be a lot more useful when you use it in context. Like selecting a chunk of text on any web page, right-click, and select summarize/translate/ELI5. Or executing your own custom prompt.

I'm building a chrome extension called SublimeGPT[1] to do exactly that. Right now, you can log in to your existing ChatGPT account, go to any page, and open a chat overlay. Next version will have the context options.

[1] https://sublimegpt.com

Re: Show HN: I made a self-hosted ChatGPT UI

#16

Post by throwaway71271 »

1xdevloper wrote: post #15

ChatGPT API can be a lot more useful when you use it in context. Like selecting a chunk of text on any web page, right-click, and select summarize/translate/ELI5. Or executing your own custom prompt. I'm building a chrome extension called SublimeGPT[1] to do exactly that. Right now, you can log in to your existing ChatGPT account, go to any page, and open a chat overlay. Next version will have the context options. [1…

you can also just use bookmarklet (or multiple defining different prompts):

    function __summarize(api_key) {
        var selection = window.getSelection().toString();
        if (selection.length == 0) return;
    
        var xhr = new XMLHttpRequest();
        xhr.open("POST", "https://api.openai.com/v1/chat/completions");
        xhr.setRequestHeader('Content-Type', 'application/json');
        xhr.setRequestHeader('Authorization', 'Bearer ' + api_key);
        window.scrollTo({top: 0})
        document.body.innerHTML = 'asking...'
        document.body.style.backgroundColor = "white";
        document.body.style.color = "black";
        document.body.style.fontFamily = 'monospace'
        document.body.style.fontSize = "16px"
        document.body.style.margin = "auto"
        document.body.style.padding = "1rem"
        document.body.style.maxWidth = "60rem"
        xhr.onreadystatechange = function() {
            if (xhr.readyState == 4) {
                if (xhr.status == 200) {
                    var response = JSON.parse(xhr.responseText);
                    var summary = response.choices[0].message.content;
                    document.body.innerHTML = summary
                } else {
                    try {
                        var e = JSON.parse(xhr.responseText);
                        document.body.innerHTML = e.error.message
                    } catch(e) {
                        document.body.innerHTML = 'error asking.. check the console'
                        console.log(xhr)
                    }
                }
            }
        }
    
        var data = JSON.stringify({
            "model": "gpt-3.5-turbo",
            "messages": [
                {"role": "system", "content": "Summarize the following text as if you are Richard Feynman"},
                {"role": "user", "content": selection}
            ]
        });
        xhr.send(data);
    }
(i have it as bookmarklet here https://gist.github.com/jackdoe/ce5a60b97e6d8487553cb00aa43f... change "YOUR API KEY HERE" with your key)

Re: Show HN: I made a self-hosted ChatGPT UI

#17

Post by tottenval »

neilv wrote: post #2

> Chat with GPT is an open-source, unofficial ChatGPT app with extra features and more ways to customize your experience. It connects ChatGPT with ElevenLabs to give ChatGPT a realistic human voice. Looks like only GUI aspects of the UI are self-hosted, but that the text and speech aspects of the UI (and the bulk of the computation and IP) are provided by two SaaS services. Self-hosted (and some degree of open) ML mo…

It's a self-hosted UI for ChatGPT right now, but my primary goal is to build a good open source chat interface that can be adapted to open source chat models as they become available.

Integrating with Alpaca, Llama, ChatGLM, OpenChatBox and whatever comes next should be straightforward once people figure out reliable and fast methods to run the models locally.

Re: Show HN: I made a self-hosted ChatGPT UI

#18

Post by 1xdevloper »

throwaway71271 wrote: post #16
1xdevloper wrote: post #15

ChatGPT API can be a lot more useful when you use it in context. Like selecting a chunk of text on any web page, right-click, and select summarize/translate/ELI5. Or executing your own custom prompt. I'm building a chrome extension called SublimeGPT[1] to do exactly that. Right now, you can log in to your existing ChatGPT account, go to any page, and open a chat overlay. Next version will have the context options. [1…

you can also just use bookmarklet (or multiple defining different prompts): function __summarize(api_key) { var selection = window.getSelection().toString(); if (selection.length == 0) return; var xhr = new XMLHttpRequest(); xhr.open("POST", "https://api.openai.com/v1/chat/completions"); xhr.setRequestHeader('Content-Type', 'application/json'); xhr.setRequestHeader('Authorization', 'Bearer ' + api_key); window.scroll…

And when you want to create/edit/delete/import custom prompts? AI is a commodity now and a great UX drives adoption.

Re: Show HN: I made a self-hosted ChatGPT UI

#20

Post by throwaway71271 »

1xdevloper wrote: post #18
throwaway71271 wrote: post #16

Earlier quoted context omitted.

you can also just use bookmarklet (or multiple defining different prompts): function __summarize(api_key) { var selection = window.getSelection().toString(); if (selection.length == 0) return; var xhr = new XMLHttpRequest(); xhr.open("POST", "https://api.openai.com/v1/chat/completions"); xhr.setRequestHeader('Content-Type', 'application/json'); xhr.setRequestHeader('Authorization', 'Bearer ' + api_key); window.scroll…

And when you want to create/edit/delete/import custom prompts? AI is a commodity now and a great UX drives adoption.

then you download 1xdeveloper's extension :)

though i just copy and paste the bookmarklet and change the prompt

Post reply on HN