Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 is no longer the only game in town

lastweekin.ai

Post reply on HN
Print view View original

21–30 of 217 posts

Re: GPT-3 is no longer the only game in town

#21

Post by phone8675309 »

13415 wrote: post #14
mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

Automatic generation of positive fake customer reviews on Amazon, landing pages about topics that redirect to attack and ad sites, fake "journalism" with auto-generated articles mixed with genuine press releases and viral marketing content, generating fake user profiles and automated karma farming on social media sites, etc. etc.

> fake "journalism" with auto-generated articles mixed with genuine press releases and viral marketing content

How would you tell the difference from the real thing these days?

Re: GPT-3 is no longer the only game in town

#22

Post by visarga »

The GPT-3 family is still to expensive to use, too big to fit in memory on a single machine. Prices need to come down before large scale adoption or someone needs to invent the chip to hold it (cheaply).

The most exciting part about it is showing us there is a path forward by scaling and prompting, but you can still do much better with a smaller model and a bit of training data (which can come from the expensive GPT-3 as well).

What I expected from the next generation: multi-modality, larger context, using retrieval to augment input data with fresh information, tuned to solve thousands of tasks with supervised data so it can generalize on new tasks better, and some efficient way to keep it up to date and fine-tune it. On the data part - more data, more languages - a lot of work.

Re: GPT-3 is no longer the only game in town

#23

Post by DeathArrow »

mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

>What would a business use it for

If you think about business uses you can actually get advices from Jerome Powell, simulated by GPT-3.

If someone use GPT-3 to simulate Warren Buffet, he can extract even more value.

https://www.institutionalinvestor.com/article/b1tktmhcfdyqsk...

Re: GPT-3 is no longer the only game in town

#24

Post by skybrian »

inglor wrote: post #20
crubier wrote: post #4

Earlier quoted context omitted.

Have you heard of GitHub copilot ? It’s based on GPT3 and I can tell you one thing: it does not generate meaningless prose (90% of the time)

This - it is tremendously valuable to me and I use it all the time at work.

What do you use it for?

Re: GPT-3 is no longer the only game in town

#25

Post by PeterisP »

DeathArrow wrote: post #9

People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a rea…

The current way of training is efficient when compute is located in a single place and is colocated with large quantities of training data. Distributing small parts of computation to remote computers is theoretically possible (and an active direction of research) but currently not preferable nor widely used; you really need very high bandwidth between all the nodes to constantly synchronize the hundreds-of-gigabytes sized weights they're iterating on and the resulting gradients.

Re: GPT-3 is no longer the only game in town

#26

Post by rewq4321 »

nathias wrote: post #19
DeathArrow wrote: post #9

People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a rea…

Yes, this is an old idea (which I really like) but it hasn't really taken off yet. GridCoin was one example, where you solved BOINC problems or RLC that's for more general computation.

The problem is that, currently, large ML models need to be trained on clusters of tightly-connected GPUs/accelerators. So it's kinda useless having a bunch of GPUs spread all over the world with huge latency and low bandwidth between them. That may change though - there are people working on it: https://github.com/learning-at-home/hivemind

Re: GPT-3 is no longer the only game in town

#27

Post by Kiro »

nathias wrote: post #19
DeathArrow wrote: post #9

People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a rea…

Yes, this is an old idea (which I really like) but it hasn't really taken off yet. GridCoin was one example, where you solved BOINC problems or RLC that's for more general computation.

It hasn't taken off because it doesn't work. PoW only works for things that are hard to calculate but easy to verify. Any meaningful result is equally hard to verify.

Re: GPT-3 is no longer the only game in town

#28

Post by DeathArrow »

amelius wrote: post #10
mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

I hope that one day it will allow me to write down my thoughts in bullet-list form, and it will then produce beautiful prose from it. Of course this will be another blow for journalists, who rely on this skill for their income.

I played with GPT-3 giving it long news stories. It actually replied with more meaningful titles than the journalists themselves used.

Re: GPT-3 is no longer the only game in town

#29

Post by DeathArrow »

13415 wrote: post #14
mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

Automatic generation of positive fake customer reviews on Amazon, landing pages about topics that redirect to attack and ad sites, fake "journalism" with auto-generated articles mixed with genuine press releases and viral marketing content, generating fake user profiles and automated karma farming on social media sites, etc. etc.

The state of the journalism is so poor, I'd rather take some AI generated articles instead.

Re: GPT-3 is no longer the only game in town

#30

Post by mrbukkake »

DeathArrow wrote: post #23
mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

>What would a business use it for If you think about business uses you can actually get advices from Jerome Powell, simulated by GPT-3. If someone use GPT-3 to simulate Warren Buffet, he can extract even more value. https://www.institutionalinvestor.com/article/b1tktmhcfdyqsk...

I can't tell whether or not this article is parody... is this a new kind of turing test
Post reply on HN