Show HN: YakGPT – A locally running, hands-free ChatGPT UI • Hacker News Board

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#31

Post by qwertox » Thu, Mar 30, 2023, 6:28 PM UTC

kami8845 wrote: post #10

FriedPickles wrote: post #8

I love the concept of this and other alternate ChatGPT UIs, but I hesitate to use them and pay for my calls when I could use chat.openai.com for free. Any chance you could integrate the backend-api, and let me paste in my Bearer token from there?

Hey! I definitely understand the reservation. This is definitely me as well. My reasons for using the UI at this point: * GPT-4 is decently faster when talking straight to the API * The API is so stupidly cheap that it's basically a rounding error for me. Half an hour of chatting to GPT3.5 costs me $0.02 Would be curious what you mean by integrating the backend-api?

GPT-3.5 is really cheap (prompt and completion = $0.002 / 1K tokens), but GPT-4 is around 20 times more expensive (prompt = $0.03 / 1K tokens + completion = $0.06 / 1K tokens).

But the benefit from using the API is that you can change the model on the fly, so you chat with 3.5 until you notice that it's not responding properly and, with all the history you have (probably stored in your database), you can send a bigger request with a probably better response once with GPT-4 as the selected model.

I really wish the interface on chat.openai.org would allow me to switch between models in the same conversation in order to 1) not use up your quota of GPT-4 interactions per 3 hours as quickly and 2) not strain the backend unnecessarily when you know that starting a conversation with GPT-3.5 is efficient enough until you notice that you better switch models.

OpenAI already has this implemented: When you use up your quota of GPT-4 chats, it offers you to drop down into GPT-3.5 in that same conversation.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#32

Post by drusepth » Thu, Mar 30, 2023, 6:35 PM UTC

robopsychology wrote: post #30

monkmartinez wrote: post #19

Earlier quoted context omitted.

You need to check which model you are using, also... LangChain runs through the model several times with increased token count on each successive call.

Yeah I assumed it would be doing several times but still more expensive than OP mentioned. I think the issue is I'm using davinci-003

Yeah, davinci-003 is gonna be gpt3, which is more expensive than 3.5.

One more anecdote: I've been running a half dozen gpt3.5 IRC bots for a few weeks and their total cost was less than a dollar. A few hours of playing around with LangChain on gpt3 cost me almost $4 before I realized I needed to switch to 3.5, though even then it still uses a ton of tokens every chain.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#33

Post by einpoklum » Thu, Mar 30, 2023, 6:38 PM UTC

Hi ChatGPT! Let me register using my personal information, then tell you what my tasks are at works, what I'm interesting in, what I'm struggling with in life and a bunch of other sensitive personal information. I trust you completely, and am sure a nice AI such as yourself would never use my personal data for anything.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#34

Post by robopsychology » Thu, Mar 30, 2023, 6:40 PM UTC

drusepth wrote: post #32

robopsychology wrote: post #30

Earlier quoted context omitted.

Yeah I assumed it would be doing several times but still more expensive than OP mentioned. I think the issue is I'm using davinci-003

Yeah, davinci-003 is gonna be gpt3, which is more expensive than 3.5. One more anecdote: I've been running a half dozen gpt3.5 IRC bots for a few weeks and their total cost was less than a dollar. A few hours of playing around with LangChain on gpt3 cost me almost $4 before I realized I needed to switch to 3.5, though even then it still uses a ton of tokens every chain.

Thanks, I'll do that later

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#35

Post by meghan_rain » Thu, Mar 30, 2023, 6:42 PM UTC

> Run locally on browser – no need to install any applications

> Please enter your OpenAI key

...

Do people just not get it?

I would in fact rather give all my company secrets to this random dude than OpenAI.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#36

Post by mthoms » Thu, Mar 30, 2023, 6:50 PM UTC

pibefision wrote: post #25

asow92 wrote: post #14

Love the idea of prompt dictation. Taking that idea a step further, would it possible to have a feature where ChatGPT responses are spoken back to the user?

War Games

"Do you want to play a game?"

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#37

Post by 1xdevloper » Thu, Mar 30, 2023, 6:59 PM UTC

FriedPickles wrote: post #8

I love the concept of this and other alternate ChatGPT UIs, but I hesitate to use them and pay for my calls when I could use chat.openai.com for free. Any chance you could integrate the backend-api, and let me paste in my Bearer token from there?

You can try the extension I built [0] which uses your existing ChatGPT session to send requests.

[0] https://sublimegpt.com

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#38

Post by jwarden » Thu, Mar 30, 2023, 7:03 PM UTC

Nice. It took about a minute to clone it, run it, enter my API key, and get started. The speech-to-text worked flawlessly.

Most people can talk faster than they can type, but they can read faster than other people can talk. So an interface where I speak but read the response is an ideal way of interfacing with ChatGPT.

What would be nice is if I didn't have to press the mic button to speak -- if it could just tell when I was speaking (perhaps by saying "hey YakGPT"). But I see how that might be hard to implement.

Would love to hook this up to some smart glasses with a heads-up display where I could speak and read the response.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#39

Post by connorgutman » Thu, Mar 30, 2023, 7:07 PM UTC

kami8845 wrote: post #11

kristopolous wrote: post #3

Make it easier to try

Hey! I would love to. I seriously considered adding my own key into the app, and implementing some rate limiting to e.g. allow you to send 3 messages for free. But unfortunately that would require me to store some backend data on you that I do not want: I want this to be a completely "private" / FE-only application that stores no data on anyone.

Testing YakGPT right now, excellent work! I would recommend adding some screenshots to the GitHub README so that people can get an idea of how it looks before entering their API key.

Re: Show HN: YakGPT – A locally running, hands-free ChatGPT UI

#40

Post by sebzim4500 » Thu, Mar 30, 2023, 7:08 PM UTC

qwertox wrote: post #31

kami8845 wrote: post #10

Earlier quoted context omitted.

Hey! I definitely understand the reservation. This is definitely me as well. My reasons for using the UI at this point: * GPT-4 is decently faster when talking straight to the API * The API is so stupidly cheap that it's basically a rounding error for me. Half an hour of chatting to GPT3.5 costs me $0.02 Would be curious what you mean by integrating the backend-api?

GPT-3.5 is really cheap (prompt and completion = $0.002 / 1K tokens), but GPT-4 is around 20 times more expensive (prompt = $0.03 / 1K tokens + completion = $0.06 / 1K tokens). But the benefit from using the API is that you can change the model on the fly, so you chat with 3.5 until you notice that it's not responding properly and, with all the history you have (probably stored in your database), you can send a bigge…

Sure, but GPT-4 through the UI costs $20 per month, which is a lot of api calls.