Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The average chess players of Bletchley Park and AI research in Britain

blogs.bl.uk

Post reply on HN
Print view View original

31–40 of 46 posts

Re: The average chess players of Bletchley Park and AI research in Britain

#31

Post by barrenko »

pyman wrote: post #8
PaulRobinson wrote: post #2

It's strange today to remember that playing chess well was seen as a great marker of AI, but today we consider it much less so. I thought Turing's Test would be a good barometer of AI, but in today's World of mountains of AI slop fooling more and more people, and ironically there being software that is better at solving CAPTCHAs than humans, I'm not so sure. Add into the mix that there are reports of people developin…

I have a similar philosophical question: My dog doesn't know what I do for a living, and he has no concept of how intelligent I am. So if we're limited by our own intelligence, how would we ever recognise or measure the intelligence of an AI that's more advanced than us? If an AI surpasses us, not just in memory or calculation but in reasoning, self-reflection, and abstraction, how would we even know?

Whatever the hell observes the mind is not the same thing as it.

Re: The average chess players of Bletchley Park and AI research in Britain

#32

Post by gowld »

Retric wrote: post #22
card_zero wrote: post #20

Earlier quoted context omitted.

Either the alleged super-intelligence affects us in some way, directly or indirectly by altering things we can detect about the world/universe, in which case we can ultimately detect it, or else it doesn't, in which case it might as well belong to a separate universe, not only in terms of our perception but objectively too. The error here is thinking that dogs understand anything.

Some dogs can respond to “bring me my slippers” and go get them in another room, a concrete task that’s still difficult for robots today. With dogs it’s less a question of intelligence but communication something more intelligent AI is unlikely to have a problem with.

Robotic legs are hard, but also pretty wells solved. See Boston Dynamics.

Re: The average chess players of Bletchley Park and AI research in Britain

#33

Post by ben_w »

PaulRobinson wrote: post #2

It's strange today to remember that playing chess well was seen as a great marker of AI, but today we consider it much less so. I thought Turing's Test would be a good barometer of AI, but in today's World of mountains of AI slop fooling more and more people, and ironically there being software that is better at solving CAPTCHAs than humans, I'm not so sure. Add into the mix that there are reports of people developin…

> and they seem - even with a lot of investment in agentic workflows and getting a lot of context into GraphRAG or wiring up MCP - to be good at helping experts get a bit faster, not replace experts. And that's not software development specific - it seems to be the case across all domains of expertise.

For now, this is a good thing: Given how generally LLMs are displacing juniors, if this was a situation where doing the same thing but harder can replace experts, it replaces approximately all of them.

But: in limited domains, not the "G" of "AGI" but just specific places here and there, AI does beat human experts. Those domains are often sufficiently narrow that they don't even encompass the entire role — think "can analyse an X-ray for cancer, can't write up its findings" kind of specificity. Indeed, I can only think of two careers where even the broadest definition of AI (some kind of programmable system) has been able to essentially fully replace that occupation:

1. https://en.wikipedia.org/wiki/Jacquard_machine

2. https://en.wikipedia.org/wiki/Computer_(occupation)

Re: The average chess players of Bletchley Park and AI research in Britain

#34

Post by Retric »

gowld wrote: post #32
Retric wrote: post #22

Earlier quoted context omitted.

Some dogs can respond to “bring me my slippers” and go get them in another room, a concrete task that’s still difficult for robots today. With dogs it’s less a question of intelligence but communication something more intelligent AI is unlikely to have a problem with.

Robotic legs are hard, but also pretty wells solved. See Boston Dynamics.

Legs are a small subset of the problem. “Where did I see that last?” involves mapping out and classifying the environment. There’s some really impressive demos of manipulating objects on a table etc, but random clutter throughout a house is still a problem.

Re: The average chess players of Bletchley Park and AI research in Britain

#35

Post by pyman »

card_zero wrote: post #28
TheOtherHobbes wrote: post #25

Earlier quoted context omitted.

Dogs certainly do understand things. Dogs and cats have a theory of mind and can successfully manipulate and trick their owners - and each other. Our perceptions are shaped by our cognitive limitations. A dog doesn't know what the Internet is, and completely lacks the cognitive capacity to understand it. An ASI would almost certainly develop some analogous technology or ability, and it would be completely beyond us.…

Dogs certainly do not understand things. Do they enquire? What are some good dog theories? They have genetic theories. We breed theories into them.

Yes, dogs can think and make choices, learn from experience, solve problems (like opening doors or finding hidden treats), and adapt to new situations.

The reason I mentioned my dog is because, even though dogs have limited intelligence compared to humans, my dog thinks he's better at playing ball than me. What he doesn't know is that I let him win because it makes him feel in control.

Re: The average chess players of Bletchley Park and AI research in Britain

#36

Post by ben_w »

pyman wrote: post #8
PaulRobinson wrote: post #2

It's strange today to remember that playing chess well was seen as a great marker of AI, but today we consider it much less so. I thought Turing's Test would be a good barometer of AI, but in today's World of mountains of AI slop fooling more and more people, and ironically there being software that is better at solving CAPTCHAs than humans, I'm not so sure. Add into the mix that there are reports of people developin…

I have a similar philosophical question: My dog doesn't know what I do for a living, and he has no concept of how intelligent I am. So if we're limited by our own intelligence, how would we ever recognise or measure the intelligence of an AI that's more advanced than us? If an AI surpasses us, not just in memory or calculation but in reasoning, self-reflection, and abstraction, how would we even know?

Depends by how much they're smarter, and in which ways.

As a trivial example, a century ago "can do arithmetic" was a signifier of being smart, yet if the entire human population today were all as fast as the current world record holder and on the same team, we would be defeated by one Raspberry Pi.

Easy to measure, but also very limited sense of "smart".

A Pi can also run Stockfish, so in that also-limited sense of "smart", it still beats humans. And chess inspires the wider use of Elo ratings in AI arenas, which means we can usefully assign scores to different AI that all beat the best humans.

For now, it's possible to point to things humans are (collectively) able to do better than AI — I originally wrote "very, very easy" rather than "possible", but then remembered noticing that whenever anyone actually tries to do so here on Hacker News, they're out of date already and there's is an AI which can do that thing superhumanly well (either that or they overstate what humans can do, e.g. claiming we can beat the halting problem); actual research papers with experiments generally do better when it comes to listing AI failure modes, including when the research comes from an AI lab showing off their new AI.

Re: The average chess players of Bletchley Park and AI research in Britain

#37

Post by GuB-42 »

PaulRobinson wrote: post #2

It's strange today to remember that playing chess well was seen as a great marker of AI, but today we consider it much less so. I thought Turing's Test would be a good barometer of AI, but in today's World of mountains of AI slop fooling more and more people, and ironically there being software that is better at solving CAPTCHAs than humans, I'm not so sure. Add into the mix that there are reports of people developin…

> I thought Turing's Test would be a good barometer of AI

Depends on what you consider a "Turing's Test".

Fooling unsuspecting humans is relatively easy, it has been done with relatively simple software and some trickery. LLMs can do that too of course.

A more convincing "Turing's Test" would be:

- You have one interrogator, and two players, one human and one computer

- The interrogator, after chatting with both players has to find which is which

- The interrogator is an expert in the field, he knows everything there is to know when it comes to finding the computer

- The human player is also an expert, he knows how to solve problems that are hard for computers to solve, he also knows what to expect from the interrogator

- The interrogator and human player collaborate to find the computer

- The interrogator and human player are not allowed to have shared information that the computer doesn't have (and ideally, they shouldn't know each other personally), but everything else is fair game

Re: The average chess players of Bletchley Park and AI research in Britain

#38

Post by NitpickLawyer »

nyrikki wrote: post #24
zmgsabst wrote: post #6

Earlier quoted context omitted.

I’d argue we have AGI, at the level of a child; now we’re debating further steps, such as adult AGI and super intelligence. But because AI is not like us, we have different results at different stages — eg, they’ve been better at arithmetic for a hundred years, games for twenty, and slowly are climbing up other domains.

Any discussion about AGI requires a written definition of the term to have a reasonable discussion. What we have now matches what many of the popular texts would call "Narrow AI", which is limited to specific tasks like speech recognition or playing chess, or mixtures of those. Traditionally AGI represents a more aspirational goal, machines that could theoretically perform any intellectual task a human can do. Under…

> Any discussion about AGI requires a written definition of the term to have a reasonable discussion.

I agree, this is the hardest thing to pin in any discussion. What version of AGI are we even talking about?

Here's a definition / explanation of AGI from the early days of DeepMind (from the movie "The thinking game"):

Quote from Shane Legg: "Our mission was to build an AGI - an artificial general intelligence, and so that means that we need a system which is general - it doesn't learn to do one specific thing. That's really key part of human intelligence, learn to do many many things".

Quote from Hassabis: "So, what is our mission? We summarise it as . So we always stress the word general and learning here the key things."

And the key slide (that I think cements the difference between what AGI stood for then, vs. now):

AI - one task vs. AGI - many tasks

at human level intelligence.

----

Now, if we go by this definition, which is pretty specific and clear, I think we've already achieved this. We already have systems that have "generally" learned stuff. And can do "many tasks" at "human level intelligence". Again, notice the emphasis on "general" and "learning". We have a learning machine, that takes in vast amounts of tokens (text, multimodal, even bytes at the end of the day) and "learns" to "do" many things. And notice it's many tasks, not all tasks. I think this is QED at this point.

But, due to the old problem of "AI is everything that hasn't been done yet", and the constant goalpost moving, together with lots and lots of writing on this topic, the waters are muddier today, and lots of people argue and emphasise different things in the AGI field.

> Fortunately there is a lot of practical utility without AGI

Yeah, completely agree. I'm with Simon's recent article on this one. It doesn't even matter at this point if we reach AGI or not, or who's definition we use. I get a lot of value today from these systems. The debates are moot from my point.

Re: The average chess players of Bletchley Park and AI research in Britain

#39

Post by AnimalMuppet »

NitpickLawyer wrote: post #38
nyrikki wrote: post #24

Earlier quoted context omitted.

Any discussion about AGI requires a written definition of the term to have a reasonable discussion. What we have now matches what many of the popular texts would call "Narrow AI", which is limited to specific tasks like speech recognition or playing chess, or mixtures of those. Traditionally AGI represents a more aspirational goal, machines that could theoretically perform any intellectual task a human can do. Under…

> Any discussion about AGI requires a written definition of the term to have a reasonable discussion. I agree, this is the hardest thing to pin in any discussion. What version of AGI are we even talking about? Here's a definition / explanation of AGI from the early days of DeepMind (from the movie "The thinking game"): Quote from Shane Legg: "Our mission was to build an AGI - an artificial general intelligence, and s…

> Now, if we go by this definition, which is pretty specific and clear...

I was going to say, no, you've defined "general" pretty well, but "intelligence" you didn't define at all. But on second thought, I guess you did - learning.

I might amend that slightly. It might be learning to do. I don't care if it can learn the words about, say, chemistry. Can it learn to solve chemistry problems?

The remaining area of fuzziness is hidden in "at human level". At what human level? I took a year of college chemistry. Can it do chemistry at that level? How about at the level of someone with a BS in chemistry? A PhD? Those are all "human" levels, but they are very different.

If it can do, say, all college subjects at undergrad level... I guess that's the benchmark for "a well rounded human".

> I think we've already achieved this.

I want to think about it some more before I definitely agree, but you've made the best case that I have seen.

The flaw I think I see is that, from a well-rounded education, we expect a human to be able to specialize, to become an expert in something. I'm not sure LLMs are quite there yet. They're closer than I was thinking 10 minutes ago, though.

Re: The average chess players of Bletchley Park and AI research in Britain

#40

Post by hyghjiyhu »

nyrikki wrote: post #24
zmgsabst wrote: post #6

Earlier quoted context omitted.

I’d argue we have AGI, at the level of a child; now we’re debating further steps, such as adult AGI and super intelligence. But because AI is not like us, we have different results at different stages — eg, they’ve been better at arithmetic for a hundred years, games for twenty, and slowly are climbing up other domains.

Any discussion about AGI requires a written definition of the term to have a reasonable discussion. What we have now matches what many of the popular texts would call "Narrow AI", which is limited to specific tasks like speech recognition or playing chess, or mixtures of those. Traditionally AGI represents a more aspirational goal, machines that could theoretically perform any intellectual task a human can do. Under…

The true definition of AGI is unfortunately "I know it when I see it". Any technical definition is provisional and achieves precision at the expense of the high risk of being simply wrong. That's not to say it lacks value. Finding how and when the provisional is wrong is itself a valuable insight bringing us closer to AGI.
Post reply on HN