Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AlphaCode as a dog speaking mediocre English

scottaaronson.blog

Post reply on HN
Print view View original

81–90 of 263 posts

Re: AlphaCode as a dog speaking mediocre English

#81

Post by skybrian »

mark_l_watson wrote: post #38

Right on, I couldn’t agree more. We are living during a period of exponential progress. I like AlphaCode’s approach of using language models with search. In the last year I have experimented with mixing language models for NLP with semantic web/linked data tasks, so much simpler than what AlphaCode does, but I have been having fun. I have added examples for this in new additions to two of my books, but if you want a…

Could you say a bit more about how you use CoPilot? What are the "other tools" you use?

Re: AlphaCode as a dog speaking mediocre English

#82

Post by kgeist »

omnicognate wrote: post #18

As someone who is skeptical, but open minded, about the impact these technologies will have on practical programming I think I'm one of the "people" in "people are complaining..." The article makes some assumptions about what such people think that certainly aren't true for me: 1. That we are unimpressed. I'm gobsmacked. 2. That we don't think these are significant advances. They're obviously huge advances. 3. That w…

>The problem is that doing so is, I expect, harder for the human than writing the code in the first place.

Programming is mostly about writing boilerplate code using well-known architectural patterns and technologies, nothing extraordinary but which takes time (at least in my experience). If I can describe a project in a few abstract words, and the AI generates the rest, it can considerably improve my productivity, and I don't think it's going to be harder to review than code written by a junior dev who also makes mistakes and doesn't usually get it right on the first try anyway (given the AI is pretrained to know what kind of architecture we prefer). I can envision a future where programmers are basically AI operators who iterate on the requirements with the client/stakeholders and let AI do the rest. It looks like we're almost there (with GitHub Copilot and all), and I think it's enough to "revolutionize" the industry, because it changes the way we approach problems and makes us far more productive with less effort.

Re: AlphaCode as a dog speaking mediocre English

#83

Post by ianbutler »

erwincoumans wrote: post #64
ianbutler wrote: post #46

Earlier quoted context omitted.

A. Most people still think Google search is good. B. Unless you work for Google specifically on that search team I'm going to say you don't know what you're talking about. So we can safely throw that point away. I've implemented a natural language search using bleeding edge work, the results I can assure you are impressive. Everything from route planning to spam filtering has seen major upgrades thanks to ML in the l…

One of Deepmind's goals is AGI, so it is tempting to evaluate their publications for progress towards AGI. Problem is, how do you evaluate progress towards AGI? https://deepmind.com/about "Our long term aim is to solve intelligence, developing more general and capable problem-solving systems, known as artificial general intelligence (AGI)."

AGI is a real problem but the proposed pace is marketing fluff -- on the ground they're just doing good work and moving our baselines incrementally. If a new technique for let's say document translation is 20% cheaper/easier to build and 15% more effective that is a breakthrough. It is not a glamorous world redefining breakthrough but progress is more often than not incremental. I'd say more so than the big eureka moments.

Dipping into my own speculation, to your point about how to measure, between our (humanity's) superiority complex and with how we move the baselines right now I don't know if people will acknowledge AGI if and until it's far superior to us. If even an average adult level intelligence is produced I see a bunch of people just treating it poorly and telling the researchers that it's not good enough.

Edit: And maybe I should amend my original statement to say I've never heard a researcher promise me about AGI. That said that statement from DeepMind doesn't really promise anything other than they're working towards it.

Re: AlphaCode as a dog speaking mediocre English

#84

Post by 37ef_ced3 »

recursive wrote: post #76
sinuhe69 wrote: post #55

I like to see how AlphaCode will solve a problem no human has solved before (or very unlikely). For example, given 3 beads and 2 stacks for the 10er and 1er positions of a number. One can make 4 different numbers when stack all the beads like in an abacus. Without using all the beads, one can of course make more numbers. The question is how many different numbers one can make using n beads, giving full and partial us…

Why do you think that's more difficult than the backspace question? I see no reason to doubt AlphaCode can solve it.

Because the backspace question (essentially: is T a subsequence of S with a deletion size of N?) probably occurs hundreds of times, in one form or another, within AlphaCode's training corpus.

Any leetcode grinder can tell you there are a few dozen types of competitive programming problem (monostack, breadth-first state search, binary search over solution space, etc.) so solutions to new problems are often very similar to solutions for old problems.

The training corpus for these code transformers is so large that almost all evaluation involves asking them to generate code from the training corpus.

To evaluate CoPilot, we should ask questions that are unusual enough they can't be answered through regurgitation of the training corpus.

What does CoPilot generate, given this prompt:

  // A Go function to set the middle six bits of an unsigned 64-bit integer to 1.
Here is a human solution:

  func six1s(x uint64) uint64 {
      const (
          ones  = uint64(63)
          shift = (64 - 6) / 2
          mask  = ones
Can it solve this simple but unusual problem?

Re: AlphaCode as a dog speaking mediocre English

#85

Post by shagie »

qnsi wrote: post #56
shagie wrote: post #45

Earlier quoted context omitted.

Bunny is a neat dog... https://www.youtube.com/channel/UCEa46rlHqEP6ClWitFd2QOQ The vocal apparatus is not there, but there is certainly more cognition than what people think dogs have (there's a question that I wonder if language enables thought or if thought enables language) And this is even something that a cat can do (not as much language skills, but there's thought going on) https://www.youtube.com/c/BilliSpeak…

I saw a lot of videos about this Bunny dog on tiktok, but discarded it as a gimmick, not believing it's real. Your comment motivated me to look into it more (30 seconds of time). This NYT article at least does not discredit it [0]. Have you looked more into it? Do you think it would be useful to train your dog to do it? [0] https://www.nytimes.com/2021/05/27/style/bunny-the-dog-anima...

With my cat (I've got the buttons, haven't done anything with them yet) its would be useful to find out if he wants food, attention, or is complaining about the water bowl or litter box.

Even being able to distinguish those would be a "win".

The Bunny video that I still find the most useful communication is https://youtu.be/6MMGmRVal6M

Re: AlphaCode as a dog speaking mediocre English

#86

Post by ma2rten »

asdfasgasdgasdg wrote: post #5
evrydayhustling wrote: post #3

I love this take. Most AI results provoke a torrent of articles listing pratfalls that prove it's not AGI. Of course it's not AGI! But it is as unexpected as a talking dog. Take a second to be amazed, at least amused. Then read how they did it and think about how to do better.

It's rarely productive to take internet criticism into account, but it feels like AI is an especially strong instance of this. It seems like a lot of folks just want to pooh pooh any possible outcome. I'm not sure why this is. Possibly because of animosity toward big tech, given big tech is driving a lot of the research and practical implementation in this area?

It's because you have to pay really close attention to tell if it's real or hype. It's really easy to make a cool demo in machine learning, cherrypick outputs, etc.

Re: AlphaCode as a dog speaking mediocre English

#87

Post by CTmystery »

Amazing stuff for sure. Looking at the example on page 59, though, I certainly see a description that contains sufficient information to implement against. I read this, and then I jump back into the tech spec that I'm writing to find:

(1) The product specification and use cases are so poorly defined that I need to anticipate the use cases, design a system that is general enough to accommodate them, and implement it in a way that is easily changeable to accommodate the future departures from my assumptions.

(2) As I do this, I need to consider the existing systems that I'm building on top of to ensure there is no regression when this feature rolls out

(3) I consider the other teams that are doing similar work and make judgement calls about whether to write independent systems that do one thing each, or to collaborate on a general-enough system with multiple team ownership.

(4) The tech that I use to implement this must be within the narrow slice of company-sanctioned tech.

(5) I weigh constant tradeoffs on speed to market, maintainability and ownership.

I'm sure there's more, but this stuff is _hard_. If autonomous driving for white collar work is coming, as put forth by comments here, I'd like to see indications that the actual hard part of the job is in jeopardy of being executed effectively.

Maybe I don't want to believe it, so I can't see it. I'll grant that. But I truly do not see it.

Re: AlphaCode as a dog speaking mediocre English

#88

Post by mark_l_watson »

skybrian wrote: post #81
mark_l_watson wrote: post #38

Right on, I couldn’t agree more. We are living during a period of exponential progress. I like AlphaCode’s approach of using language models with search. In the last year I have experimented with mixing language models for NLP with semantic web/linked data tasks, so much simpler than what AlphaCode does, but I have been having fun. I have added examples for this in new additions to two of my books, but if you want a…

Could you say a bit more about how you use CoPilot? What are the "other tools" you use?

I installed the VSCode and PyCharm CoPilot plugins, and signed in with my GitHub account (you first need to request CoPilot access and wait until you get it).

As you type in comments or code, CoPilot will sometime autocomplete up to about 10 lines of code, based on the content of the file you are editing (maybe just the code close to the edit point?).

My other tools? I use LispWorks Professional, Emacs with Haskell support, and sometimes IntelliJ. I work a lot on remote servers so I depend on SSH/Mosh and tmux also.

Re: AlphaCode as a dog speaking mediocre English

#89

Post by baron816 »

Fun fact: dogs bark primarily to communicate with humans. Wolves (undomesticated dogs) don’t really bark. And you wouldn’t likely see a pack of dog barking at each other. But humans are keenly able to tell what a dog is trying to express by the different sounds it makes. This is all a result of the convolution between the two species.

Re: AlphaCode as a dog speaking mediocre English

#90

Post by EGreg »

_vufv wrote: post #62
EGreg wrote: post #41

No, just no! Because the whole thing is, a dog’s abstract mental capabilities are far below a human — thus why it would be ASTOUNDING that a dog could master even a primitive form of speaking English. On the other hand, here we are brute forcing a solution from analyzing millions of man-years of published English speech, by using a huge array of computing power to precompute various answers and sift them. It is a bit…

Excuse my presumption, but it seems that you arrive at the logical conclusion "it might be possible to simulate intelligence with a sufficiently big cheat sheet" - and then you disregard it because you're uncomfortable with it. We already know this is the case for specialized environments, so the "only" question left is how far does this generalize. In my opinion, more ridiculous claims have already been proven by sc…

I don’t disregard it. It isn’t about discomfort. In fact, I think that “solving checkers” is very useful, if your goal is to get the highest quality answers in checkers.

The problem I have is comparing that to having a dog speak English. It’s totally wrong. You had access to all these computing resources, and the sum total of millions of work by humans. You didn’t bootstrap from nothing like AlphaZero did, but just remixed all possible interesting combinations, then selected the ones you liked. And you try to compare this “top down” approach to a bottom-up one?

The top down approach may give BETTER answers and be MORE intelligent. But the way it arrives at this is far less impressive. In fact, it would be rather expected.

Post reply on HN