Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 is no longer the only game in town

lastweekin.ai

Post reply on HN
Print view View original

101–110 of 217 posts

Re: GPT-3 is no longer the only game in town

#101

Post by j45 »

iamcurious wrote: post #57
ComodoHacker wrote: post #51

Are we heading to the (distant) future where to make progress in any field you have to spend big $$$ to train a model?

We are already there. Machine learning is the flavor of A.I. that keeps business barriers of entry high. If we had invested in symbolic A.I., things would be different. A similar thing happens with programming language flavors. PHP lowers barriers of entry so it is discredited by the incumbents.

Your point about incumbents not wanting it to be easier to create beginners in a language or technology is very understated.

Excluding participation in having the time and resources available to overcome the initial inertia required to become productive is a form of opportunity and earning segregation.

Despite having a background in your tech, there is little more if satisfying than people experiencing putting tech to work for them, rather than the other way around or being dependent on others.

Re: GPT-3 is no longer the only game in town

#102

Post by Nextgrid »

minimaxir wrote: post #92
ComodoHacker wrote: post #51

Are we heading to the (distant) future where to make progress in any field you have to spend big $$$ to train a model?

Fortunately, costs for training superlarge models are coming down rapidly thanks to TPUs (which was the approach used to train GPT-J 6B) and DeepSpeed improvements.

Are there any TPUs that can be purchased off-the-shelf and then owned, like you can do with a CPU or GPU? Or are you just limited to paying rent to cloud providers and ultimately being at their mercy when it comes to pricing, ToS, etc?

Re: GPT-3 is no longer the only game in town

#103

Post by worik »

The underlying methods seem impractical. GPT-n are an existence proof - it is possible to make parrot like software that generates realistic text. But using these methods it is not practical.

Maybe that is a good thing, maybe a bad thing, but unless there is a breakthrough in methods this is a dead end. Impressive though.

Re: GPT-3 is no longer the only game in town

#104

Post by teaearlgraycold »

supermatt wrote: post #56
qwertox wrote: post #47

If I were to run GPT-3 on my 70000 browser bookmarks, what kind of insights could I get from that? Only by analyzing the page title (from the bookmark, not by re-fetching the url) and eventually also the domain name.

GPT-3 is a text generator, so i doubt you would get anything of use. You cant even supply such a large input to GPT-3.

GPT-3 is also a classifier and data-extractor.

You could give it a couple dozen bookmarks with example classifications and then feed it a new bookmark and ask GPT-3 what category the page belongs in. Repeat for the entire data set.

For data extraction you could ask questions about the titles. Maybe have it list all machine-learning model names that appear in the set of bookmark titles.

Re: GPT-3 is no longer the only game in town

#105

Post by teaearlgraycold »

mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

I work for a company that re-sells GPT-3 to small business owners. We help them generate product descriptions in bulk, Google ads, Facebook ads, Instagram captions, etc.

Re: GPT-3 is no longer the only game in town

#106

Post by Nextgrid »

DeathArrow wrote: post #9

People were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a rea…

My understanding is that proof-of-work is intentionally wasteful; the objective is to make 51% attacks (where a single entity controls at least 51% of the global hashrate) infeasible by attaching a cost to the mining process.

Making the mining process produce useful output that can be resold nullifies the purpose as it means an attacker can now mine "for free" as a byproduct of doing general-purpose computations (as opposed to tying up dedicated hardware), lowering the barrier for a 51% attack dramatically.

Re: GPT-3 is no longer the only game in town

#107

Post by teaearlgraycold »

akelly wrote: post #68
mrbukkake wrote: post #2

Can anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for

https://copy.ai/

Hey, I work there! To be honest it's still very much a prototype. We have big plans for the next few months.

Re: GPT-3 is no longer the only game in town

#108

Post by teaearlgraycold »

sva_ wrote: post #34
DeathArrow wrote: post #23

Earlier quoted context omitted.

>What would a business use it for If you think about business uses you can actually get advices from Jerome Powell, simulated by GPT-3. If someone use GPT-3 to simulate Warren Buffet, he can extract even more value. https://www.institutionalinvestor.com/article/b1tktmhcfdyqsk...

> Interviewer: Are you in favor of a carbon tax? > Mr. Powell: I don’t want to get into the details of taxes. > Interviewer: Are you in favor of a cap and trade system? > Mr. Powell: I don’t want to get into the details of a cap and trade system. > Interviewer: How do you think a global pandemic would impact the financial system? > Mr. Powell: A pandemic will have impact on the economy. It will have an impact on the…

It does pretty well at transforming text into a person's style of talking. So you could have it re-write any sentence to sound like a Trump tweet.

Re: GPT-3 is no longer the only game in town

#109

Post by snovv_crash »

Kiro wrote: post #27
nathias wrote: post #19

Earlier quoted context omitted.

Yes, this is an old idea (which I really like) but it hasn't really taken off yet. GridCoin was one example, where you solved BOINC problems or RLC that's for more general computation.

It hasn't taken off because it doesn't work. PoW only works for things that are hard to calculate but easy to verify. Any meaningful result is equally hard to verify.

It's easy to verify ML training - inference on a test set has lower error than it did before.

Training NN ML is much slower than inference (1000x at least) because it has to calculate all of the gradients.

Re: GPT-3 is no longer the only game in town

#110

Post by WithinReason »

worik wrote: post #103

The underlying methods seem impractical. GPT-n are an existence proof - it is possible to make parrot like software that generates realistic text. But using these methods it is not practical. Maybe that is a good thing, maybe a bad thing, but unless there is a breakthrough in methods this is a dead end. Impressive though.

https://copilot.github.com/
Post reply on HN