Weak-to-Strong Generalization

I assume much less than one correction per word. Corrections are to fix when the kids made a mistake when they mimic others, kids doesn't make mistakes for every word so you don't need to correct every word.

As I said, bulk of learning is from mimicking, but they will make a lot of mistakes when mimicking so to become good they need corrections. When I see parents with kids I see them correcting their kids all the time, correcting misunderstandings or fixing pronunciations or grammar issues etc. You can notice that parents use a specific voice when they correct the pronunciation, and the kid gets that the parent tried to correct their pronunciation from it, that happens quite a lot with small kids. You don't say "this is how to pronounce X", our genes seem to encode a way to communicate pronunciation to others to help correct mistakes.

Kids also correct each other in this way, which helps make the learning more robust even when adults aren't around. All humans has instincts to correct others so I don't think any culture fails to do this.

This may or may not be true about the weather but is definitely not true in general so fails as an analogy for your argument. Lots of functions are invertable or partially invertable and if you think about ML as discovering a function, then training on the outputs, learning the inverse function, then inverting that (numerically or otherwise) to discover the true function is certainly doable for some problems.

On the stock market there are for sure people who predict the markets by training on the outputs of the price. The (weak) efficient markets hypothesis is enough to say the price reflects all the exogenous information that is available to the market about the stock so you don't need all the fundamentals etc and there are lots of people who trade in that way.

The history of language models in computing is that research started by building very complex systems that attempted to encode "how the world works" by developing very intricate rule systems and build linguistic/semantic models from there. These "expert systems" tended towards being arcane and brittle and lacked the ability to reason outside their ruleset in general. They also tended to reveal gaps/cracks in our understanding of how language works etc.

Yes, factual knowledge across generations accumulates, also being lost, too. However, even if it's not lost, the theory still holds true.

Nature is evolving, everything gets better over time. From bacterium to apes to humans. We evolve, accumulate knowledge and being able to build more sophisticated machinery, or tame more complex processes to build more sophisticated things. Even bacteria transfers memories across generations.

However, this doesn't remove the ceiling. Total human knowledge will always be larger and deeper than any A.I. we can create, because the upper limit is always what we can consciously manipulate and put into something. Your next car has the possibility to contain more technology, because we can build more complex factories to manufacture them. Yet, a car can't be more complex/sophisticated than its factory.

Consider a semiconductor fab. You can use the output of that fab to design/create better fab, but the process needs human intervention. Invention of new things are generally necessary. Better processes, optics, hardware, etc.

Another nice example is reprap machines. A reprap can print all plastic parts required for the machine. You need to get metal parts yourself, and assemble them. If you want to be able to print metal parts, machine gets more complicated. So a reprap which can build itself completely is at least sophisticated as the resulting reprap itself, but you need to hand assemble it again.

If you want an assembling reprap, now that thing becomes a factory. Again, the complexity of the product is at most the same as the building machine. You can create better factories which has more streamlined processes, but the gap widens again. The factory becomes more complex than the output.

As a human, you're the factory. Your upper limit is another human. You can create things more complex than a human by using multiple humans, but the creator ends up more complex than the creature itself.

You're moving up the ceiling, that's true, but everything we build is capped by our collective capacity. That's the truth.

A.I. is glorified search. It can wander in the box you create for it and show places you missed inside it, but can't show something outside that box.