Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why don't machine learning research agents overfit?

amazon.science

Post reply on HN
Print view View original

21–30 of 70 posts

Re: Why don't machine learning research agents overfit?

#21

Post by bee_rider »

cj wrote: post #18
serial_dev wrote: post #14

Earlier quoted context omitted.

Time to first detected slop in this article is Is it too much to ask from people to read their own article anymore? If anyone read this at all, they would have had the ick, and would have fired off a prompt to get rid of the most popular AI slop tells...

What I really dislike is having to edit my own non-LLM assisted writing to make sure I'm not accidentally confused with AI. I caught myself writing "And that matters because..." in a HN comment but had to edit myself. Also miss uising emdashes.

These models are trained on human language, which belongs to us, we shouldn’t surrender it to them. Keep the em-dashes. IMO don’t overuse negative parallelisms though, they were always bad and lazy.

Re: Why don't machine learning research agents overfit?

#22

Post by beckhamc »

diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

And sadly, in academia, complexity (opposite of Occam's razor) is what gets you published.

Re: Why don't machine learning research agents overfit?

#23

Post by srean »

diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

I think you should get less annoyed.

> It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple.

I don't know what Occam meant, but if you accept the formalism of PAC learning, it is more likely to be correct

https://web.archive.org/web/20170428225156/http://www.cse.bu...

https://web.archive.org/web/20130412062821/http://cs.ecs.bay...

Re: Why don't machine learning research agents overfit?

#24

Post by p-e-w »

srean wrote: post #23
diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

I think you should get less annoyed. > It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. I don't know what Occam meant, but if you accept the formalism of PAC learning, it is more likely to be correct https://web.archive.org/web/20170428225156/http://www.cse.bu... https://web.archive.org/web/20130412062821/http://cs.ecs.bay...

There are also various metaphysical theories that posit that the universe is algorithmically generated in some sense or the other, and from many of those theories it follows that simplicity is a fundamental feature of reality, which yields an even stronger version of Occam’s Razor.

Re: Why don't machine learning research agents overfit?

#28

Post by diddid »

gowld wrote: post #13
diddid wrote: post #8

I always get annoyed when people misinterpret Occam’s razor. It’s not that the simplest is more likely to be correct, it’s that you should prefer it, because it’s simple. It’s just like the Hopper quote. She said it’s better to ask for forgiveness during the fog of war, doing something you thought was right, not to do something you knew they were going to say no to and now you are trying to get away with something.

That's not true. It's pretty clear that she meant "do something you knew they were going to say no to and now you are trying to get away with something." https://youtu.be/wHdHCoeUbU4?t=861s > So I want to tell something to all the young people here on many many occasions you'll find it is much easier to apologize than it is to get permission. You do it then when somebody comes after you and say are you supposed to do…

But I still don’t think that means eat all the cookies in the cookie jar and then apologize after because nobody would have given permission. That’s still about doing what you believe to be right. She even frames the fallout as “where you supposed to do that?” and not “you shouldn’t have done that”.

Re: Why don't machine learning research agents overfit?

#29

Post by srean »

bee_rider wrote: post #21
cj wrote: post #18

Earlier quoted context omitted.

What I really dislike is having to edit my own non-LLM assisted writing to make sure I'm not accidentally confused with AI. I caught myself writing "And that matters because..." in a HN comment but had to edit myself. Also miss uising emdashes.

These models are trained on human language, which belongs to us, we shouldn’t surrender it to them. Keep the em-dashes. IMO don’t overuse negative parallelisms though, they were always bad and lazy.

An arms race on style would be interesting. Essentially a real life GAN.
Post reply on HN