Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-3 is no longer the only game in town

lastweekin.ai

Post reply on HN
Print view View original

111–120 of 217 posts

Re: GPT-3 is no longer the only game in town

#111

Post by DonHopkins »

iamcurious wrote: post #57
ComodoHacker wrote: post #51

Are we heading to the (distant) future where to make progress in any field you have to spend big $$$ to train a model?

We are already there. Machine learning is the flavor of A.I. that keeps business barriers of entry high. If we had invested in symbolic A.I., things would be different. A similar thing happens with programming language flavors. PHP lowers barriers of entry so it is discredited by the incumbents.

PHP wasn't discredited by the incumbents. It was discredited by its creator.

"I'm not a real programmer. I throw together things until it works then I move on. The real programmers will say Yeah it works but you're leaking memory everywhere. Perhaps we should fix that. I'll just restart Apache every 10 requests." -Rasmus Lerdorf

"I was really, really bad at writing parsers. I still am really bad at writing parsers." -Rasmus Lerdorf

"We have things like protected properties. We have abstract methods. We have all this stuff that your computer science teacher told you you should be using. I don't care about this crap at all." -Rasmus Lerdorf

Re: GPT-3 is no longer the only game in town

#112

Post by CodeGlitch »

lostdog wrote: post #91
iamcurious wrote: post #57

Earlier quoted context omitted.

We are already there. Machine learning is the flavor of A.I. that keeps business barriers of entry high. If we had invested in symbolic A.I., things would be different. A similar thing happens with programming language flavors. PHP lowers barriers of entry so it is discredited by the incumbents.

The difference between ML and symbolic AI is that ML works and symbolic AI doesn't. At my job, dropping the computational load of our ML models is heavily invested in, and every success is celebrated. Everybody wants it to be easier and cheaper to train high quality models, but some things are still intrinsically hard.

> The difference between ML and symbolic AI is that ML works and symbolic AI doesn't.

IBM managed to beat Garry Kasperov using symbolic AI did they not? So in what way does it not work?

Re: GPT-3 is no longer the only game in town

#113

Post by rm_-rf_slash »

DeathArrow wrote: post #28
amelius wrote: post #10

Earlier quoted context omitted.

I hope that one day it will allow me to write down my thoughts in bullet-list form, and it will then produce beautiful prose from it. Of course this will be another blow for journalists, who rely on this skill for their income.

I played with GPT-3 giving it long news stories. It actually replied with more meaningful titles than the journalists themselves used.

Perhaps GPT-3 was optimizing to deliver information while news sites these days optimize titles to get clicks.

Re: GPT-3 is no longer the only game in town

#114

Post by cma »

kingcharles wrote: post #89
cma wrote: post #80

Earlier quoted context omitted.

How do you avoid libel?

Are you confusing libel with something else? Can you extrapolate what you mean here? Are you saying that they will be liable for libel (!) if they publish a negative summary of a product?

If they mischaracterize a positive review into a negative summary based on factual mistakes they know the system makes at a high rate, I would think they would be liable for libel right?

Re: GPT-3 is no longer the only game in town

#115

Post by ALittleLight »

pharmakom wrote: post #83
crubier wrote: post #4

Earlier quoted context omitted.

Have you heard of GitHub copilot ? It’s based on GPT3 and I can tell you one thing: it does not generate meaningless prose (90% of the time)

Code is easier to write than read and maintain, so how useful is something that generates pages of 90% correct code?

It's not useful if you use it to auto complete pages of code. It is useful to see it propose lines, read, and accept its proposals. Sometimes it just saves you a second of typing. Sometimes it makes a suggestion that causes you to update what you wanted to do. Sometimes it proposes useless stuff. On the whole, I really like it and think it's a boon to productivity.

Re: GPT-3 is no longer the only game in town

#116

Post by amelius »

rg111 wrote: post #93

The future is not as dark as it seems because of the rat race of megacorps. You can use reduced versions of language models with extremely good results. I was involved in training the first-ever GPT2 for Bengali language, but with 117 million parameters. It took a month's effort (training + writing code + setup) and about $6k in TPU cost, but Google Cloud covered it. Anyway, it is surprisingly good. We fine-tuned the…

> The future is not as dark as it seems because of the rat race of megacorps.

Just wait until NVidia comes with a "Neural AppStore" and corresponding restrictions. Then wait until the other GPU manufacturers follow suit.

Re: GPT-3 is no longer the only game in town

#117

Post by cmrajan »

rg111 wrote: post #93

The future is not as dark as it seems because of the rat race of megacorps. You can use reduced versions of language models with extremely good results. I was involved in training the first-ever GPT2 for Bengali language, but with 117 million parameters. It took a month's effort (training + writing code + setup) and about $6k in TPU cost, but Google Cloud covered it. Anyway, it is surprisingly good. We fine-tuned the…

Good to know. We're trying to attempt something similar[1] but for Tamil. I'm also surprised how well the OSS language model & library AI4Bharat [2] performs for NLP tasks against SoTA systems. Is there a way to contact you? [1] https://vpt.ai/posts/about-us/ [2] https://ai4bharat.org/projects/

Re: GPT-3 is no longer the only game in town

#118

Post by adgjlsfhk1 »

CodeGlitch wrote: post #112
lostdog wrote: post #91

Earlier quoted context omitted.

The difference between ML and symbolic AI is that ML works and symbolic AI doesn't. At my job, dropping the computational load of our ML models is heavily invested in, and every success is celebrated. Everybody wants it to be easier and cheaper to train high quality models, but some things are still intrinsically hard.

> The difference between ML and symbolic AI is that ML works and symbolic AI doesn't. IBM managed to beat Garry Kasperov using symbolic AI did they not? So in what way does it not work?

they didn't. that was just alpha beta search with some custom hardware to speed it up. also at this point, both of the strongest chess ai (stockfish and lc0) are using neutral networks and are roughly 1000 elo above where deep blue was (and most of that is from software, not hardware)

Re: GPT-3 is no longer the only game in town

#119

Post by [deleted] »

supperburg wrote: post #52

I have supported my beliefs on this topic in these threads to the point of exhausting myself. The tools that we use to find these agents are the underpinning of AGI, it’s coming way faster than even most people here appreciate, this development is intrinsically against the interest of human beings. Please stop and think, please.

[deleted]
Post reply on HN