Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can I download GPT / ChatGPT to my desktop?

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 51 posts

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#21

Post by cal85 »

madiator wrote: post #15
turkeygizzard wrote: post #7

I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able…

Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.

Wouldn’t that be possible with about 4 powerful GPUs? Or does it not work like that?

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#24

Post by xaml »

PM_me_your_math wrote: post #12
luckyme123 wrote: post #8

Luckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)

Rename it and ask it to find & kill the virus it created?

I don't think that I can PM you, but could you explain how Green's theorem can be applied to regions with singularities?

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#25

Post by ghilston »

navjack27 wrote: post #23

You can do gpt-j https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4... I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2

Do you have any writeups on what prompt engineering you've done to get gpt-J to behave like a chat?

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#26

Post by AHappyCamper »

5e92cb50239222b wrote: post #3

OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use: https://en.wikipedia.org/wiki/GPT-3

Truly a shame. Good marketing tho

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#27

Post by dragonwriter »

There are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”.

You probably won't be able to run (or especially train) them on typical desktops, though.

[0] https://www.eleuther.ai/projects/gpt-neox/

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#28

Post by consumer451 »

5e92cb50239222b wrote: post #3

OpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use: https://en.wikipedia.org/wiki/GPT-3

I think a more accurate name might be Alignment AI. [0]

I realize it's a charitable interpretation of their behavior, and I used to be very angry about them not being truly open. However, after playing with ChatGPT I think I am beginning to understand and even support their behavior. [1]

My personal sea change was realizing that giving dual-use tools to the global @everyone and hoping for the best might not be the greatest plan. I came to this realization thinking about bio-tech and GNC software, but it may apply to ML products as well. [2]

While I used to think universally/religiously "it should be open, will be one day anyway, etc," I now think about these things on a case by case basis.

[0] https://openai.com/alignment/

[1] https://news.ycombinator.com/item?id=33928400

[2] Imagine how many nodes this bot farm would have if it wasn't limited by the existing C&C bottleneck. ChatGPT is a productivity multiplier. This is productivity I want to put off multiplying as long as possible: https://news.ycombinator.com/item?id=34165350

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#29

Post by puffybuf »

Sharlin wrote: post #10
turkeygizzard wrote: post #7

I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able…

I don't think ChatGPT does. I have had long discussions with it, with some rules agreed upon in the beginning, and at some point it clearly begins to forget the exact rules and has to be reminded of them. (Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To…

I read that it just re-reads the discussion so far every time you submit. So it must hit a limit of what it can remember since they limit the amount of training tokens it can read for a submission.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#30

Post by Sharlin »

puffybuf wrote: post #29
Sharlin wrote: post #10

Earlier quoted context omitted.

I don't think ChatGPT does. I have had long discussions with it, with some rules agreed upon in the beginning, and at some point it clearly begins to forget the exact rules and has to be reminded of them. (Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To…

I read that it just re-reads the discussion so far every time you submit. So it must hit a limit of what it can remember since they limit the amount of training tokens it can read for a submission.

Yes, I know. It’s a pure function with no mutable state.
Post reply on HN