Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Can I download GPT / ChatGPT to my desktop?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 51 posts

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#41

Post by throwaway2016a »

Sharlin wrote: post #10
turkeygizzard wrote: post #7

I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able…

I don't think ChatGPT does. I have had long discussions with it, with some rules agreed upon in the beginning, and at some point it clearly begins to forget the exact rules and has to be reminded of them. (Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To…

Is Chat-GTP it's own model? I thought ChatGTP was just GTP-3 with an easier to use interface.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#42

Post by Sharlin »

throwaway2016a wrote: post #41
Sharlin wrote: post #10

Earlier quoted context omitted.

I don't think ChatGPT does. I have had long discussions with it, with some rules agreed upon in the beginning, and at some point it clearly begins to forget the exact rules and has to be reminded of them. (Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To…

Is Chat-GTP it's own model? I thought ChatGTP was just GTP-3 with an easier to use interface.

It's based on GPT-3 but is specifically amended to predict sequences that look like coherent dialogue, by an adversarial model that has been partially trained by humans. The resulting model is also quite a bit smaller than the full GPT-3. It's much more difficult to make GPT-3 engage in reasonable dialogue than ChatGPT.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#43

Post by taocoyote »

cal85 wrote: post #21
madiator wrote: post #15

Earlier quoted context omitted.

Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.

Wouldn’t that be possible with about 4 powerful GPUs? Or does it not work like that?

Possibly, but that would be 10 of thousands of dollars worth of GPUs.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#44

Post by dilap »

throwaway2016a wrote: post #40
dilap wrote: post #16

Earlier quoted context omitted.

It's really obnoxious for non-open companies to include "open" in the name. Stolen valour, basically.

Maybe if you compare it to open source. But if you consider that many of these algorithms were historically invented by a private company and kept private / proprietary as a competitive advantage I think the fact OpenAI puts no unreasonable restrictions on who can use it makes it fairly "open" Though, I too, would rather be able to run the model myself.

Publishing research is pretty normal for traditional companies. The pledge to make patents available is a small step towards open.

But I think the obvious and natural thing for a company in the business of training machine learning models that claims to be open to do is make the models themselves available.

OpenAI does rougly the opposite: not only do you not have acccess to the underlying models, even the API access you are given is to a model itself deliberately trained to avoid answering certain classes of queries.

To me that's the opposite of open; it's closed, restricted, and centrally controlled.

(Very impressive results tho!)

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#45

Post by rzzzt »

fred967 wrote: post #38

You can download his ancestor here: https://winworldpc.com/product/dr-sbaitso/2x

  HELLO FRED967,  MY NAME IS DOCTOR SBAITSO.
  
  I AM HERE TO HELP YOU.
  SAY WHATEVER IS IN YOUR MIND FREELY,
  OUR CONVERSATION WILL BE KEPT IN STRICT CONFIDENCE.
  MEMORY CONTENTS WILL BE WIPED OFF AFTER YOU LEAVE,
  
  SO, TELL ME ABOUT YOUR PROBLEMS.
That falling intonation is very reassuring.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#47

Post by joshka »

madiator wrote: post #15
turkeygizzard wrote: post #7

I'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able…

Yeah it wouldn't fit. GPT3 is 175B params, so even if you use 8 bit for each weight, you need 175×10^9÷2^30 = 163GiB of memory.

https://www.reddit.com/r/ChatGPT/comments/zhzjpq/comment/izo...

>It's around 500gbs and requires around 300+gbs of vram from my understanding and runs on one of the largest super computers in the world. Sable diffusion has around 6 billion parameters gpt-3/chatgpt has 175 billion.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#48

Post by throwaway2016a »

dilap wrote: post #44
throwaway2016a wrote: post #40

Earlier quoted context omitted.

Maybe if you compare it to open source. But if you consider that many of these algorithms were historically invented by a private company and kept private / proprietary as a competitive advantage I think the fact OpenAI puts no unreasonable restrictions on who can use it makes it fairly "open" Though, I too, would rather be able to run the model myself.

Publishing research is pretty normal for traditional companies. The pledge to make patents available is a small step towards open. But I think the obvious and natural thing for a company in the business of training machine learning models that claims to be open to do is make the models themselves available. OpenAI does rougly the opposite: not only do you not have acccess to the underlying models, even the API access…

> Publishing research is pretty normal for traditional companies.

Published research is "open" to the extent that it is transparent but it is not "open" to the extent that it can be used and accessed by people. Unless you are an AI researcher, half these papers (to be generous) might as well not exist.

My argument is from that perspective (ability for the average person to use it), academic research only gives the illusion of openness.

Not only that but the training data is often -- but not always -- omitted from academic research. So reproducing the exact results they did is often out of reach without a significant investment in building your own collection of training data.

For example: Facebook and Google have both announced similar technology to OpenAI yet neither is usable out of the box (or at all for practical purposes) where as OpenAI despite being "closed" I can get started in 5 minutes.

Take by contrast to both of those, Stable Diffusion. Which I think is miles ahead of DALL-E... their code and their pre-trained weights are very easy to use as well as being open.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#49

Post by PM_me_your_math »

xaml wrote: post #24
PM_me_your_math wrote: post #12

Earlier quoted context omitted.

Rename it and ask it to find & kill the virus it created?

I don't think that I can PM you, but could you explain how Green's theorem can be applied to regions with singularities?

It was a (bad) SkyNet joke, quoting the third terminator movie.

Re: Ask HN: Can I download GPT / ChatGPT to my desktop?

#50

Post by dilap »

throwaway2016a wrote: post #48
dilap wrote: post #44

Earlier quoted context omitted.

Publishing research is pretty normal for traditional companies. The pledge to make patents available is a small step towards open. But I think the obvious and natural thing for a company in the business of training machine learning models that claims to be open to do is make the models themselves available. OpenAI does rougly the opposite: not only do you not have acccess to the underlying models, even the API access…

> Publishing research is pretty normal for traditional companies. Published research is "open" to the extent that it is transparent but it is not "open" to the extent that it can be used and accessed by people. Unless you are an AI researcher, half these papers (to be generous) might as well not exist. My argument is from that perspective (ability for the average person to use it), academic research only gives the il…

All good points. The cambrian explosion of stable diffusion variants & uses is a good demonstration of the benefits of a more open system.
Post reply on HN