Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-J: GPT-3 Democratized

p3r.one

Post reply on HN
Print view View original

1–10 of 41 posts

Re: GPT-J: GPT-3 Democratized

#4

Post by zitterbewegung »

I run GPT-J on a Titan RTX where I am writing a novel with it. To make it generate about 20k tokens or two pages of content takes a few minutes . I would say the output is comparable to other language models quality and so forth.

Note that refinement or transfer learning doesn’t apply anymore it’s more like using a zero shot classifier or in other words you have to craft the input like Siri or wolfram alpha but expect text back instead

It runs on about 15gb of VRAM and https://www.eleuther.ai/ released it under the Apache license.

I use this endpoint written by kinoc using FastAPI https://gist.github.com/kinoc/f3225092092e07b843e3a2798f7b39... which is released under the MIT licence.

Re: GPT-J: GPT-3 Democratized

#6

Post by Tenoke »

rorykoehler wrote: post #3

Dupe https://news.ycombinator.com/item?id=27727009

Yeah, twice in one day (and 3+ times in a month) is a bit much and this post adds little new except some oddities like singling kubernetes in particular as 'the technology that helped train GPT-3'.

Re: GPT-J: GPT-3 Democratized

#7

Post by mmastrac »

zitterbewegung wrote: post #4

I run GPT-J on a Titan RTX where I am writing a novel with it. To make it generate about 20k tokens or two pages of content takes a few minutes . I would say the output is comparable to other language models quality and so forth. Note that refinement or transfer learning doesn’t apply anymore it’s more like using a zero shot classifier or in other words you have to craft the input like Siri or wolfram alpha but expec…

Can I run it on an RTX-3090? Where would I find information on how to do it?

EDIT: Reply below pointed out that the gist linked above specifically mentions 3090 at the top.

Re: GPT-J: GPT-3 Democratized

#8

Post by Tenoke »

mmastrac wrote: post #7
zitterbewegung wrote: post #4

I run GPT-J on a Titan RTX where I am writing a novel with it. To make it generate about 20k tokens or two pages of content takes a few minutes . I would say the output is comparable to other language models quality and so forth. Note that refinement or transfer learning doesn’t apply anymore it’s more like using a zero shot classifier or in other words you have to craft the input like Siri or wolfram alpha but expec…

Can I run it on an RTX-3090? Where would I find information on how to do it? EDIT: Reply below pointed out that the gist linked above specifically mentions 3090 at the top.

The link mentions Titan and 3090 specifically so I'd guess yes.
Post reply on HN