Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaCode as a dog speaking mediocre English

scottaaronson.blog

Post reply on HN
Print view View original

31–40 of 263 posts

Re: AlphaCode as a dog speaking mediocre English

#31

Post by sabujp »

Isinlor wrote: post #27
turminal wrote: post #9

It would be interesting to know how much of this improvement in the last 25 since he was a student comes from Moore's law, other hardware improvements, various new technologies not related to AI, amount of money being thrown at the problem... and how much of it are advancements in our understanding of AI.

Roughly half/half, skewed more towards improvements in algorithms. Measuring the Algorithmic Efficiency of Neural Networks: https://arxiv.org/abs/2005.04305 In this work, we argue that algorithmic progress has an aspect that is both straightforward to measure and interesting: reductions over time in the compute needed to reach past capabilities. We show that the number of floating-point operations required to train a…

All these deepmind and openai computer scientists don't pay themselves, you're not taking their pay into consideration.

Re: AlphaCode as a dog speaking mediocre English

#32

Post by [deleted] »

stephc_int13 wrote: post #10
asdfasgasdgasdg wrote: post #5

Earlier quoted context omitted.

It's rarely productive to take internet criticism into account, but it feels like AI is an especially strong instance of this. It seems like a lot of folks just want to pooh pooh any possible outcome. I'm not sure why this is. Possibly because of animosity toward big tech, given big tech is driving a lot of the research and practical implementation in this area?

AI has been over-hyped, that's all. The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software. But people have different and irrational expectations about AI.

[deleted]

Re: AlphaCode as a dog speaking mediocre English

#33

Post by sillysaurusx »

sanxiyn wrote: post #25
stephc_int13 wrote: post #10

Earlier quoted context omitted.

AI has been over-hyped, that's all. The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software. But people have different and irrational expectations about AI.

I mean, it is magical, in a sense that we are not sure how and why it works.

I'm not sure how it works, but I'm sure it doesn't think. Not till it can choose its own loss function.

Re: AlphaCode as a dog speaking mediocre English

#34

Post by Misdicorl »

qualudeheart wrote: post #30

Which rate of progress are we ITT expecting from alphacode-like models? In another thread I predicted we’d see competitive programming “solved” in ten years or less. I didn’t rigorously explain what I meant by that in that thread, so I’ll clarify what I meant. I expect AI to beat humans at competitive programming at the same rate as AlphaGo beats human Go players. There could be diminshing returns soon but I don’t se…

The places AI excel still have very rigid rules with relatively low branching factors. It's also relatively easy to score both the final result and any intermediate stage. And all the training techniques rely on those factors tremendously.

My guess is that's where the difficulties will arise.

Re: AlphaCode as a dog speaking mediocre English

#35

Post by sanxiyn »

sillysaurusx wrote: post #33
sanxiyn wrote: post #25

Earlier quoted context omitted.

I mean, it is magical, in a sense that we are not sure how and why it works.

I'm not sure how it works, but I'm sure it doesn't think. Not till it can choose its own loss function.

Sure, it doesn't think, just as submarines don't swim, as EWD said.

Re: AlphaCode as a dog speaking mediocre English

#36

Post by sillysaurusx »

sanxiyn wrote: post #35
sillysaurusx wrote: post #33

Earlier quoted context omitted.

I'm not sure how it works, but I'm sure it doesn't think. Not till it can choose its own loss function.

Sure, it doesn't think, just as submarines don't swim, as EWD said.

People always reach for these analogies. "Planes don't fly like birds." "Submarines don't swim like fish."

Backpropagation has zero creativity. It's an elaborate mechanical parrot, and nothing more. It can never relate to you on a personal level, because it never experiences the world. It has no conception of what the world is.

At least a dog gets hungry.

Not persuaded? Try https://news.ycombinator.com/item?id=23346972

Re: AlphaCode as a dog speaking mediocre English

#37

Post by ipaddr »

ianbutler wrote: post #21
stephc_int13 wrote: post #10

Earlier quoted context omitted.

AI has been over-hyped, that's all. The machine learning techniques that were developed and enhanced during the last decade are not magical, like any other machines/software. But people have different and irrational expectations about AI.

Has it been over hyped? Some ML created in the last 8 years is in most major products now. It has been transformative even if you don’t see it, is informing most things you use. We’re not close to AGI but I’ve never heard an actual researcher make that claim or the orgs they work for. They just consistently show for the tasks they pick they beat most baselines and in a lot of cases humans. The models just don’t gener…

It is overhyped because in the end it has failed to deliver the breakthrough promised years ago. Car drinking cars are a great example.

The AI that has taken over google search has made a great product kinda of awful now.

What breakthroughs are you referring to?

Re: AlphaCode as a dog speaking mediocre English

#38

Post by mark_l_watson »

Right on, I couldn’t agree more. We are living during a period of exponential progress. I like AlphaCode’s approach of using language models with search. In the last year I have experimented with mixing language models for NLP with semantic web/linked data tasks, so much simpler than what AlphaCode does, but I have been having fun. I have added examples for this in new additions to two of my books, but if you want a 90 second play-time, here is a simple colab notebook https://colab.research.google.com/drive/1FX-0eizj2vayXsqfSB2...

We live in exciting times. Given exponential rates of progress, I can’t really even imagine what breakthroughs we will see in the next year, let alone the next five years.

EDIT: I forgot to mention: having GitHub/OpenAI tools like CoPilot always running in the background in PyCharm and VSCode has in a few short months changed my workflow, for the better.

Re: AlphaCode as a dog speaking mediocre English

#39

Post by isomel »

sillysaurusx wrote: post #33
sanxiyn wrote: post #25

Earlier quoted context omitted.

I mean, it is magical, in a sense that we are not sure how and why it works.

I'm not sure how it works, but I'm sure it doesn't think. Not till it can choose its own loss function.

Are you sure that you are thinking? Can you choose your loss function?

Re: AlphaCode as a dog speaking mediocre English

#40

Post by [deleted] »

ipaddr wrote: post #37
ianbutler wrote: post #21

Earlier quoted context omitted.

Has it been over hyped? Some ML created in the last 8 years is in most major products now. It has been transformative even if you don’t see it, is informing most things you use. We’re not close to AGI but I’ve never heard an actual researcher make that claim or the orgs they work for. They just consistently show for the tasks they pick they beat most baselines and in a lot of cases humans. The models just don’t gener…

It is overhyped because in the end it has failed to deliver the breakthrough promised years ago. Car drinking cars are a great example. The AI that has taken over google search has made a great product kinda of awful now. What breakthroughs are you referring to?

[deleted]
Post reply on HN