> but "proving a conjecture" might have a lower intrinsic complexity than "putting together a good joke"
Might earwax be soon worth more than gold? Experts say: No! What? No.
> "Pre-coded algebra" was thought to require a lot of intelligence too - until someone found a way to make simple logic gates perform addition, multiplication and division.
No. Not intelligence. Diligence. https://en.wikipedia.org/wiki/Computer_(occupation)
They didn't hire the smartest to do the calculations. They hired the diligent and cheap. They hired the smartest to do math.
Things that are easy for humans are easy because we have fist sized universal approximator in our skulls, that's just fast enough to keep most of us on two feet, architecturally optimized for very few activities (mostly physical, some virtualized) and trained for years. It doesn't mean things we do are complex.
As for Moravec's paradox ... Guidance system of a missile is not super smart or solving complex problems. It's just brutally optimized for the task and has a fitting form factor. Tasks that are easy for it are hard or impossible for my windows computer and vice versa. Paradox comes from stupidly thinking easyhard is one dimensional axis. That kind of thinking is something people are very prone to ... goodevil, healthysick, youngold ... while if we go a bit beyond the simplest narratives we can plainly see that everything is a multidimensional landscape. Just because we chose to draw a single line through it, in a semi-random direction we feel is about right, doesn't mean it is relevant for solving anything or even interesting. That's where a lot of paradoxes come from. We just strayed from reality too far and simplified or abstracted something too much.
> But judging intrinsic complexity of a task by whether humans find it hard is the most treacherous thing -
I think we can get a good hang of estimating how hard a thing is. If a thing is hard for a human it's probably pretty hard. We made some of them easy building machines that exceeded human strength and diligence. Now we built first one that exceed human intelligence. On one hand, it's as big as invention of a lever, steam machine or a computer. On the other hand it might be only roughly as important as those things.
... If a thing is easy for human it still might be hard because of hardware optimizations that humans have. Walking on two legs, seems easy. Walking on two arms. Much harder. But truly they are one and the same thing for a robot. So you might easily estimate that walking is not that easy. It's just when it comes to legs humans have a specialized controller, like a missile guidance system. Putting together a good joke? Might seem easy, maybe it's not that easy because humor plays a role in reproductions so we might have some optimization for it, but it's surely not harder than putting together quantum theory. You can see this from whatever the ideas version of cyclomatic complexity is. Some math theories have higher complexity than quantum theory. So a system that's capable of exploring multidimensional landscape of mathematic language, surely has raw capability of doing everything else humans can do with language. And it will once we direct it towards it correctly.
> "you can't do anything harder with your intelligence than math". This kind of statement has an awful track record.
I don't agree it has. And I stand by it.