Live data from Hacker News

Richard Sutton and Andrew Barto Win 2024 Turing Award

awards.acm.org

101–110 of 119 posts

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#101

Earlier quoted context omitted.

I am still hoping AI progress will get to the point where the AI can eventually create AI's that are built up out of robust and provable logic which can be read and audited. Until that time, I wouldn't trust it for risky stuff. Unfortunately, it's not my choice and within a scarily short timespan, black boxes will make painfully wrong decisions about vital things that will ruin lives.

Quis custodiet ipsos custodes? https://en.m.wikipedia.org/wiki/Quis_custodiet_ipsos_custode... excerpt of the first few paragraphs, sorry about any wrong formatting, links becoming plain text, etc. just pasted it as is: Quis custodiet ipsos custodes? is a Latin phrase found in the Satires (Satire VI, lines 347–348), a work of the 1st–2nd century Roman poet Juvenal. It may be translated as "Who will guard the guards t…

Apologies for taking the phrase in a slightly farcical (& incurious ?) direction:

   Who will take custody of the custodians?

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#102

Earlier quoted context omitted.

I was an undergrad interning in a computer vision lab in the early 2010s. During group meeting, someone presented a new paper that was using abstract machine learning like stuff to do vision. The prof was so visibly perturbed and agnostic. He could not believe that this approach was even a little bit viable, when it so clearly was. Best lesson for me - vowed never to be the person opposed to new approaches that work.

> Best lesson for me - vowed never to be the person opposed to new approaches that work. I think you'll be surprised at how hard that will be to do. The reason many people feel that way is because: (a) they've become an expert (often recognized) in the old approach. (b) They make significant money (or something else). At the end of the day, when a new approach greatly encroaches into your way of life -- you'll likely…

Of course it is difficult, for precisely the reasons you indicate. It's one of those lifetime skills that you have to continuously polish, and if you fall behind it is incredibly hard to recover. But such skills are necessary for being a resilient person.

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#103

Earlier quoted context omitted.

Yes, I have a small team of (me being 1/3) doing formal verification in my company and we do this and it doesn't actually matter if how the AI got there; we can mathematically say it's correct which is what matters. We do (and did) program synthesis and proofs but this is all very far from doing anything serious at scale.

What kind of company needs formal verification? Real time systems?

Companies designing digital circuits use it all the time.

Say you have a module written in VHDL or Verilog and it is passing regressions and everyone is happy. But as the author, you know the code is kind of a mess and you want to refactor the logic. Yes, you can make your edits and then run a few thousand directed tests and random regressions and hope that any error you might have made will be detected. Or you can use formal verification and prove that the two versions of your source code are functionally identical. And the kicker is it often takes minutes to formally prove it, vs hundreds to thousands of CPU hours to run a regression suite.

At some point the source code is mapped from a RTL language to gates, and later those gates get mapped to a mask set. The software to do that is complex and can have bugs. The fix is to extract the netlist from the masks and then formally verify that the extracted netlist matches the original RTL source code.

If your code has assertions (and it should), formal verification can be used to find counter examples that disprove the assertion.

But there are limitations. Often logic is too complex and the proof is bounded: it can show that from some initial state no counter example can be found in, say, 18 cycles, but there might be a bug that takes at least 20 cycles to expose. Or it might find counter examples and you find it arises only in illegal situations, so you have to manually add constraints to tell it which input sequences are legal (which often requires modeling the behavior of the module, and that itself can have bugs...).

The formal verifiers that I'm familiar with are really a collection of heuristic algorithms and a driver which tries various approaches for a certain amount of time before switching to a different algorithm to see if that one can crack the nut. Often, when a certain part of the design can be proven equivalent, it aids in making further progress, so it is an iterative thing, not a simple "try each one in turn". The frustrating thing is you can run formal on a module and it will prove there are no violations with a bounded depth of, say, 32 cycles. A week later a new release of your formal tool comes out with bug fixes and enhancements. Great! And now that module might have a proof depth of 22 cycles, even though nothing changed in the design.

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#104
post #91
post #80

Earlier quoted context omitted.

> all of their lofty accomplishments get nullified by anyone I don't think it's a question of whether their achievements are nullified, but as you mention, how to weight the opinions of various people. Personally, I think both a Turing award for technical achievement and a view that humanity ought to be replaced are relevant in evaluating someone's opinions on AI policy, and we shouldn't forget the latter because of…

By "view that humanity," do you mean alignment with the effective altruism cult? Repressive laws on open AI/models—giving elites total control in the name of safety? And this alternative perspective from the cult should disqualify someone from a Turing Award despite their achievements?

No, a "view that humanity ought to be replaced" is Sutton's, not an EA view. I'm not quite sure how you read that otherwise, except that you seem very angry. I sure hope our alternatives are better than human extinction or total control by elites...

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#105

Very cool to see this! It turns out my wife and I bought Andy Barto’s (and his wife’s) house. During the process, there was a bidding war. They said “make your prime offer” so, knowing he was a mathematician, we made an offer that was a prime number :-) So neat to see him be recognized for his work.

> we made an offer that was a prime number

$12345678910987654321?

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#106

These guys are great but unfortunately the ai sutton and barto book is really bad. You would do better with Grokking Machine Learning by trask, and then a couple months of implementing ml papers.

I second this suggestion. Read Grokking Deep Reinforcement Learning before reading Sutton. Well, the Sutton book is free, so take a peak, but if the formulas scare you then read Grokking Deep Reinforcement Learning.

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#107

These guys are great but unfortunately the ai sutton and barto book is really bad. You would do better with Grokking Machine Learning by trask, and then a couple months of implementing ml papers.

These books are about different topics? Sutton and Barto is about Reinforcement learning, and the other book you mention by Trask is on Deep Learning?

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#108

Earlier quoted context omitted.

This a somewhat uninteresting matter of semantics, but I think brute force generally refers to exhaustive search. MCTS is not brute force for that very reason (the vast majority of branches are never searched at all).

OK, but I think it's generally understood that exhaustive search is not feasible for games like Chess and Go, so when "brute force" is used in this context it means an emphasis on deep search and number of positions evaluated rather than the human approach where many orders of magnitude less positions are evaluated.

I think that kind of erodes the meaning of the phrase. A typical MCTS run for alphazero would evaluate what, like 1024 rollouts? Maybe less? That's a drop in the ocean compared to the number of states available in chess. If you call that brute force then basically everything is.

I've personally viewed well over a hundred thousand rollouts in my training as a chess bot =P

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#109

Earlier quoted context omitted.

>this fetishism to dig into another person’s personal life and find the most weird thing they said as the thing that paints over all of their life’s achievements as evil must stop. This has nothing to do with his professional life. He has made these comments in a professional capacity at an industry AI conference... The rest of your comment is a total non sequitur. >And worst of all internet gives your opinion the sa…

It just feels like a smear on his character: Imagine working on RL incrementally without any lofty goals or preconceived evil. I do agree that there is some level of inherent safety issues with such technologies - but look at atomic bomb vs fission reactors etc: history paves a way through positivity. Just because someone had an idea that eventually turned to have some evil branch off way further from the root idea d…

People left careers in AI in the 1990s because they came to realize that the tech would probably eventually become dangerous. Many more (including the star student in my CS program in the 1980s) never started a career in AI for the same reason.

Sutton and everyone else who has advanced the field deserve condemnation IMO, not awards.

Re: Richard Sutton and Andrew Barto Win 2024 Turing Award

#110
post #86

Earlier quoted context omitted.

>this fetishism to dig into another person’s personal life and find the most weird thing they said as the thing that paints over all of their life’s achievements as evil must stop. This has nothing to do with his professional life. He has made these comments in a professional capacity at an industry AI conference... The rest of your comment is a total non sequitur. >And worst of all internet gives your opinion the sa…

> This has nothing to do with his professional life. you mean his personal life?

oops, yes.
Post reply on HN