Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

EURISKO Lives

blog.funcall.org

Post reply on HN
Print view View original

51–60 of 102 posts

Re: EURISKO Lives

#51

Post by famouswaffles »

YeGoblynQueenne wrote: post #49
famouswaffles wrote: post #47

Earlier quoted context omitted.

That was a figure of speech. I didn't literally mean games (not that GOFAI performs better than NNs in those games anyway). I simply went off your own examples - Vision, Image generation, Translation etc. >As I say in my comment above planning and scheduling, SAT, constraints, verification, theorem proving- those are still dominated by classical systems You can use NNs for all these things. It wouldn't make a lot of…

I don't understand your comment. Clarify. As it is, your comment seems to tell me that neural nets are good at neural net things and GOFAI is good at GOFAI things, which is obvious, and is what I'm saying: neural nets can make only very limited use of expert knowledge and so suck in all domains where domain knowledge is abundant and abundantly useful, which are the same domains where GOFAI dominates. GOFAI can make v…

NNs can do the things GOFAI is good at a whole lot better than GOFAI can do the things NNs are good at.

Re: EURISKO Lives

#52

Post by lisper »

adw wrote: post #38
lisper wrote: post #36

Earlier quoted context omitted.

> the combination of the two still seems worth investigating This. Back in the late 1980's and early 90's the debate-du-jour was between deliberative and reactive control systems for robots. I got my Ph.D. for simply saying that the entire debate was based on the false premise that it had to be one or the other, that each approach had its strengths and weaknesses, and that if you just put the two together the whole w…

People have advanced that argument a lot, and it's often worked for a short while; then the statistical models get better. Chess was a game for humans. It was very briefly a game for humans and machines (Kasparov had a go at getting "Advanced Chess" off the ground as a competitive sport), but soon enough having a human in the team made the program worse. But at least the evaluation functions were designed by humans,…

> then the statistical models get better

Maybe. The statistical models are definitely better at natural language processing now, but they still fail on analytical tasks.

Of course, human brains are statistical models, so there's an existence proof that a sufficiently large statistical model is, well, sufficient. But that doesn't mean that you couldn't do better with an intelligently designed co-processor. Even humans do better with a pocket calculator, or even a sheet of paper, than they do with their unaided brains.

Re: EURISKO Lives

#53

Post by YeGoblynQueenne »

lisper wrote: post #52
adw wrote: post #38

Earlier quoted context omitted.

People have advanced that argument a lot, and it's often worked for a short while; then the statistical models get better. Chess was a game for humans. It was very briefly a game for humans and machines (Kasparov had a go at getting "Advanced Chess" off the ground as a competitive sport), but soon enough having a human in the team made the program worse. But at least the evaluation functions were designed by humans,…

> then the statistical models get better Maybe. The statistical models are definitely better at natural language processing now, but they still fail on analytical tasks. Of course, human brains are statistical models, so there's an existence proof that a sufficiently large statistical model is, well, sufficient. But that doesn't mean that you couldn't do better with an intelligently designed co-processor. Even humans…

If human brains are statistical models, why are human brains so bad at statistics?

Edt: btw, same for probabilistic inference, same for logical inference, and same for any other thing anyone's tried as the one true path to AI since the 1950's. Humans have consistently proven bad at everything computers are good at, and that tells us nothing about why humans are good at anything (if, indeed, we are). Let's not assume too much about brains until we find the blueprint, eh?

Re: EURISKO Lives

#54

Post by YeGoblynQueenne »

adw wrote: post #50
famouswaffles wrote: post #44

Earlier quoted context omitted.

>Two, all the loud successes of statistical machine learning in the last couple of decades are closely tied to minutely specialised neural net architectures: CNNs for image classification, LSTMs for translation, Transformers for vision, Difussion models and Ganns for image generation. If that's not encoding knowledge of a domain, what is? Transformers, Diffusion for Vision, Image generation are really odd examples he…

Yeah, all of those architectures are _themselves_ hacks to get around having insufficient compute! They absolutely were encoding inductive biases into the network to get around not being able to train enough, and transformers (handwaving hard enough to levitate, the currently-trainable model family with the least inductive bias) have eaten the world in all domains. This is evidence _for_ the Bitter Lesson, not agains…

They haven't (eaten the world etc). They just happen to be the models that trend hard right now. I bet if you could compare like for like you'd be able to see some improvement in performance from Transformers, but that 'd be extremely hard to separate from the expected improvement from the constantly increasing amounts of data and compute. For example, you could, today, train a much bigger and deeper Multi-Layered Perceptron than you could thirty years ago, but nodoy is trying because that's so 1990's, and in any case they have the data and compute to train much bigger, much more inefficient (contrary to what you say if I got that right) architectures.

Wait a few years and the Next Big Thing in AI will come along, hot on the heels of the next generation of GPUs, or tensor units or whatever the hardware industry can cook up to sell shovels for the gold rush. By then, Transfomers will have hit the plateau of diminishing returns, there'll be gold in them there other hills and nobody would talk of LLMs anymore because that's so 2020s. We've been there so many times before.

Re: EURISKO Lives

#55

Post by adw »

YeGoblynQueenne wrote: post #42
adw wrote: post #38

Earlier quoted context omitted.

People have advanced that argument a lot, and it's often worked for a short while; then the statistical models get better. Chess was a game for humans. It was very briefly a game for humans and machines (Kasparov had a go at getting "Advanced Chess" off the ground as a competitive sport), but soon enough having a human in the team made the program worse. But at least the evaluation functions were designed by humans,…

That's the "bitter lesson", right? Which is really a sour lesson- as in sour grapes. See, Rich Sutton's point with his Bitter Lesson is that encoding expert knowledge only improves performance temporarily, which is eventually surpassed by more data and compute. There are only two problems with this: One, statistical machine learning systems have an extremely limited ability to encode expert knowledge. The language of…

> The language of continuous functions is alien to most humans and it's very difficult to encode one's intuitive, common sense knowledge into a system using that language

In other words; machine learned models are octopus brains (https://www.scientificamerican.com/article/the-mind-of-an-oc...) and that creeps you out. Fair enough, it creeps me out too, and we should honour our emotions — I'm no rationalist – but we should also be aware of the risks of confusing our emotional responses with reality.

Re: EURISKO Lives

#56

Post by lisper »

YeGoblynQueenne wrote: post #53
lisper wrote: post #52

Earlier quoted context omitted.

> then the statistical models get better Maybe. The statistical models are definitely better at natural language processing now, but they still fail on analytical tasks. Of course, human brains are statistical models, so there's an existence proof that a sufficiently large statistical model is, well, sufficient. But that doesn't mean that you couldn't do better with an intelligently designed co-processor. Even humans…

If human brains are statistical models, why are human brains so bad at statistics? Edt: btw, same for probabilistic inference, same for logical inference, and same for any other thing anyone's tried as the one true path to AI since the 1950's. Humans have consistently proven bad at everything computers are good at, and that tells us nothing about why humans are good at anything (if, indeed, we are). Let's not assume…

> why are human brains so bad at statistics?

That depends on what you mean by being "bad at statistics." What brains do on a conscious level is very different than what they do at a neurobiological level. Brains are "bad at statistics" on the conscious level, but at the level of neurobiology that's all they do.

As an analogy, consider a professional tennis or baseball player. At the neurobiological level those people are extremely good at finding solutions to kinematic equations, but that doesn't mean that they would ace a physics test.

Re: EURISKO Lives

#57

Post by dang »

Related. Others?

Doug Lenat's sources for AM (and EURISKO+Traveller?) found in public archives - https://news.ycombinator.com/item?id=38413615 - Nov 2023 (9 comments)

Eurisko Automated Discovery System - https://news.ycombinator.com/item?id=37355133 - Sept 2023 (1 comment)

Why AM and Eurisko Appear to Work (1983) [pdf] - https://news.ycombinator.com/item?id=28343118 - Aug 2021 (17 comments)

Early AI: “Eurisko, the Computer with a Mind of Its Own” (1984) - https://news.ycombinator.com/item?id=27298167 - May 2021 (2 comments)

Some documents on AM and EURISKO - https://news.ycombinator.com/item?id=18443607 - Nov 2018 (10 comments)

Why AM and Eurisko Appear to Work (1983) [pdf] - https://news.ycombinator.com/item?id=9750349 - June 2015 (5 comments)

Why AM and Eurisko Appear to Work (1984) [pdf] - https://news.ycombinator.com/item?id=8219681 - Aug 2014 (2 comments)

Eurisko, The Computer With A Mind Of Its Own - https://news.ycombinator.com/item?id=2111826 - Jan 2011 (9 comments)

Let's reimplement Eurisko - https://news.ycombinator.com/item?id=656380 - June 2009 (25 comments)

Eurisko, The Computer With A Mind Of Its Own - https://news.ycombinator.com/item?id=396796 - Dec 2008 (13 comments)

Re: EURISKO Lives

#58

Post by YeGoblynQueenne »

famouswaffles wrote: post #51
YeGoblynQueenne wrote: post #49

Earlier quoted context omitted.

I don't understand your comment. Clarify. As it is, your comment seems to tell me that neural nets are good at neural net things and GOFAI is good at GOFAI things, which is obvious, and is what I'm saying: neural nets can make only very limited use of expert knowledge and so suck in all domains where domain knowledge is abundant and abundantly useful, which are the same domains where GOFAI dominates. GOFAI can make v…

NNs can do the things GOFAI is good at a whole lot better than GOFAI can do the things NNs are good at.

That's wishful thinking not supported by empirical results.

Re: EURISKO Lives

#59

Post by YeGoblynQueenne »

lisper wrote: post #56
YeGoblynQueenne wrote: post #53

Earlier quoted context omitted.

If human brains are statistical models, why are human brains so bad at statistics? Edt: btw, same for probabilistic inference, same for logical inference, and same for any other thing anyone's tried as the one true path to AI since the 1950's. Humans have consistently proven bad at everything computers are good at, and that tells us nothing about why humans are good at anything (if, indeed, we are). Let's not assume…

> why are human brains so bad at statistics? That depends on what you mean by being "bad at statistics." What brains do on a conscious level is very different than what they do at a neurobiological level. Brains are "bad at statistics" on the conscious level, but at the level of neurobiology that's all they do. As an analogy, consider a professional tennis or baseball player. At the neurobiological level those people…

That is a very big assumption -that brains have conscious and subconscious levels that are good and bad at different things- that needs to be itself proved, before it can be used to support any other line of inquiry.

I'm not well versed in the relevant literature at all but my understanding is that research in the area points to the completely opposite direction: that humans e.g. playing baseball do not find solutions to kinematic equations, but instead use simple heuristics that exploit our senses and body configuration, like placing their hands in front of their eyes so that they line up with the ball etc.

This makes a lot more sense, not only for humans playing tennis, but for animals surviving in the wild, finding sustenance and shelter, and mates, while avoiding becoming a meal. Consider the Portia spider [1], a spider-hunting spider, itself prey to other hunting spiders, with a brain consisting of a few tens of thousands of neurons and still perfectly capable not only of navigating complex environments in all three space dimensions but also making complex plans involving detours.

Just think of how quickly a spider must be able to think that hunts, and is hunted by other spiders -some of the most deadly predators in the animal kingdom. There is no chance of a snowball in hell that such an animal has the time to solve kinematic equations with a few KBs of neurons. Absolutely no chance at all.

For that and many other stuff like that it looks very unlikely to me that human brains, or any brains, are like you say. In any case, that sounds positively Freudian and I don't mean that as an insult, but I so could.

______________

[1] My favourite. No, I don't mean meal. I just love this paper; it's almost the best paper in autonomous robotics and planning that I've ever read:

https://www.frontiersin.org/journals/psychology/articles/10....

Re: EURISKO Lives

#60

Post by lisper »

YeGoblynQueenne wrote: post #59
lisper wrote: post #56

Earlier quoted context omitted.

> why are human brains so bad at statistics? That depends on what you mean by being "bad at statistics." What brains do on a conscious level is very different than what they do at a neurobiological level. Brains are "bad at statistics" on the conscious level, but at the level of neurobiology that's all they do. As an analogy, consider a professional tennis or baseball player. At the neurobiological level those people…

That is a very big assumption -that brains have conscious and subconscious levels that are good and bad at different things- that needs to be itself proved, before it can be used to support any other line of inquiry. I'm not well versed in the relevant literature at all but my understanding is that research in the area points to the completely opposite direction: that humans e.g. playing baseball do not find solution…

> That is a very big assumption -that brains have conscious and subconscious levels that are good and bad at different things- that needs to be itself proved, before it can be used to support any other line of inquiry.

You can't be serious. Do you really doubt that hand-eye coordination and solving systems of kinematic equations on paper using math are disjoint skills? That one can be good at one without being good at the other? That there is in actual fact an inverse correlation between these skills? How do you account for the fact that even people who have never studied math or physics can learn to throw and catch a ball?

Post reply on HN