Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaCode as a dog speaking mediocre English

scottaaronson.blog

Post reply on HN
Print view View original

231–240 of 263 posts

Re: AlphaCode as a dog speaking mediocre English

#231

Post by runarberg »

killerstorm wrote: post #213

A lot of people who are skeptical about AI progress call it "statistical modeling" and point to large data sets involved and large amounts of hardware thrown at it. Sort of implying it's some sort of a brute-force trick. I'm afraid they do not understand the size of problem/solution set. Suppose problem and solution are 1000 characters long and there's a set of 32 characters. Then a model is function F: X -> X where…

We are still inferring a best fit based on available data, so “statistical modelling” is way more descriptive of what we are doing the “intelligence”. That doesn’t mean it isn’t impressive. These non-classical statistical inference are going beyond what we have been able to do with classic statistics, it is indeed impressive.

Personally I only grow skeptical when claims are made about some vague Artificial General Intelligence or some other magic which no amount of inference is gonna give us.

Re: AlphaCode as a dog speaking mediocre English

#232

Post by MauranKilom »

meken wrote: post #176
telotortium wrote: post #171

Earlier quoted context omitted.

Unfortunately, GPT-3, CoPilot, AlphaCode, etc., also excel more at the creative part (helped out by their enormous training database, which saves you from doing the Google search to match your high-level description to examples), but they're still dogshit at editing, because that is the part that actually requires a detailed understanding of the text. So the princess is still in the next castle.

Hmm it intuitively seems like the editing part should be the easy part, because the output is basically a template, and you just need to fill it in, and tweak it a little bit… I get with code it’s a bit less so, but still, there are plenty of design patterns which basically cover a lot of the output space…

The context window of GPT-3 is vanishingly small compared to any novel-like work. GPT-3 will happily invent descriptions of characters and objects that contradict what it suggested three pages ago.

Not saying it can't help you with getting an idea what to describe, but this is the kind of "editing" that it won't be good at.

Re: AlphaCode as a dog speaking mediocre English

#233

Post by MauranKilom »

danielovichdk wrote: post #144

Why do we write code ? Code is not a goal. It's a tool. The tool can get better. But it will always be a tool and someone has to control the tool. Writing code is super easy. It's the easiest part of our job. The hard part of our jobs is to understand why we need to write code and right along that, what we write code for ? Maintenance Structuring Domain knowledge Refactoring Those are hard things to do. And they are…

> It's the easiest part of our job.

Yep. Some efficiency gain is surely possible with "intelligent" autocomplete (i.e. codex instead of intellisense), but it must be code you wanted to write.

"If you think you're paying me for typing code, maybe you should have hired a typist and not a programmer?"

Re: AlphaCode as a dog speaking mediocre English

#234

Post by mirceal »

Smoosh wrote: post #215
mirceal wrote: post #169

this is cool and all but it’s not even close to AGI. it’s a sophisticated gimmick. to go meta, I don’t believe that our way of approaching AI in general is going to get us to AGI. Why? Assume for a second that we could build something that could solve ANY problem. What would happen next? What does an intelligent human with zero emotions do? Nothing. Fear of dying and the hardwired desire to pass on our genes (or to c…

I agree with your underlying premise - without stimuli such as pain and pleasure to give motivation it is difficult to foresee an AI which will think enough like humans to be useful or that we feel we can trust. But perhaps that is just chauvinism.

generalized problem solving could be useful but it’s still going to be a tool that augments us. it will not replace us by any means.

the game changer is not going to be AGI, it’s going to be human/machine symbiosis

Re: AlphaCode as a dog speaking mediocre English

#235

Post by gspr »

alex-2k1 wrote: post #57
trabant00 wrote: post #15

People think more of what we already have is going to go farther. 1 horse to the carriage gets you to the market. 2 horses to the next village. 4 to town and 6 cross states. Given enough horses we should reach the moon, right? With absolutely no evidence (as none can be had about the future) I believe that AI can be reached with computers and programming languages as different from the current ones as rockets are to…

Scientists in 50s expected to get language translation in 10 years as soon as computers will get enough computation power. They were real scientists, not "data AI scientists" who has little mathematics culture and not aware of any brain studies and problems in this field. But yeah, all aboard is hype train, we have a dog who speak English! Not a state machine that just do similar to what it was programmed on using st…

While I echo some of these sentiments, I don't think they apply in the slightest to the author of the blog post this thread discusses.

Certainly a "real scientist" in every sense.

Re: AlphaCode as a dog speaking mediocre English

#236

Post by zarzavat »

skybrian wrote: post #129
mark_l_watson wrote: post #88

Earlier quoted context omitted.

I installed the VSCode and PyCharm CoPilot plugins, and signed in with my GitHub account (you first need to request CoPilot access and wait until you get it). As you type in comments or code, CoPilot will sometime autocomplete up to about 10 lines of code, based on the content of the file you are editing (maybe just the code close to the edit point?). My other tools? I use LispWorks Professional, Emacs with Haskell s…

Do you usually find the autocomplete from CoPilot useful enough to keep without editing, or is it wrong often enough to be annoying?

For me both are true. The suggestions are usually annoying and wrong, however sometimes copilot makes an observation that is non-obvious, i.e it suggests code that makes me realize that I was about to write a subtle bug. I keep it on as a sanity check.

Re: AlphaCode as a dog speaking mediocre English

#237

Post by baron816 »

robocat wrote: post #185
baron816 wrote: post #89

Fun fact: dogs bark primarily to communicate with humans. Wolves (undomesticated dogs) don’t really bark. And you wouldn’t likely see a pack of dog barking at each other. But humans are keenly able to tell what a dog is trying to express by the different sounds it makes. This is all a result of the convolution between the two species.

> convolution https://en.wikipedia.org/wiki/Coevolution I am not sure I would like to see dogs convolved with humans.

Must’ve been autocorrect

Re: AlphaCode as a dog speaking mediocre English

#238

Post by runarberg »

falcor84 wrote: post #60
amusedcyclist wrote: post #26

Earlier quoted context omitted.

I mean I'm not so impressed, because it seems like someones figured out the ventriloquist trick and and is just spamming it to make anything talk. Its fun enough, but unclear what this is achieving

>ventriloquist trick I don't understand your analogy here. It really is the machine talking; there is no human hiding behind the curtain.

I guess there is a bunch of data hiding behind the curtains, and there is human feeding the data to the model. However I don’t agree with GP here as a ventriloquist’s dummy is not doing anything a human can’t. A well trained language model can produce an output in seconds what it takes human weeks to.

Re: AlphaCode as a dog speaking mediocre English

#239

Post by _kulang »

killerstorm wrote: post #213

A lot of people who are skeptical about AI progress call it "statistical modeling" and point to large data sets involved and large amounts of hardware thrown at it. Sort of implying it's some sort of a brute-force trick. I'm afraid they do not understand the size of problem/solution set. Suppose problem and solution are 1000 characters long and there's a set of 32 characters. Then a model is function F: X -> X where…

I would think of it as fitting a function which is over-parameterised. So there is intrinsically a model of how things are thought to behave, it’s just a relatively simple one that is over-parameterised so it captures patterns in the data. Get it wrong and it’s over-fit, get it wrong another way and it doesn’t generalise, etc. There can also be an internal representation which is not easily interpretable, an internal representation which is usually of a dimensionality much lower than the number of parameters.

It is in this sense that it is a brute-force approach because we should like to know the underlying model, but instead we can throw a hugely over-parameterised but relatively simple model at a problem and it will learn (statistically) the underlying phenomenon. Like you say it does much better than a combinatorial brute-force approach.

Re: AlphaCode as a dog speaking mediocre English

#240

Post by moyix »

ahgamut wrote: post #189
37ef_ced3 wrote: post #84

Earlier quoted context omitted.

Because the backspace question (essentially: is T a subsequence of S with a deletion size of N?) probably occurs hundreds of times, in one form or another, within AlphaCode's training corpus. Any leetcode grinder can tell you there are a few dozen types of competitive programming problem (monostack, breadth-first state search, binary search over solution space, etc.) so solutions to new problems are often very simila…

> To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus. Exactly! It's great that Copilot can generate correct code for a given question, but we cannot gauge its full capability unless we try it on a range of different questions, especially ones that are not found in the training data. I mentioned this in the other AlphaCode post: It w…

AlphaCode does some analysis of training data copying in their paper (Sections 6.1 and Appendix F): https://storage.googleapis.com/deepmind-media/AlphaCode/comp...

It does not seem to be copying from the training data in any meaningful way.

Post reply on HN