Cargo Cult AI

>Because we don't think one word at a time

In what sense does an LLM think one word at a time that doesn't also apply to a person typing at a keyboard? I'm typing one word at a time right now, I assume you aren't about to declare me a markov chain. When I read my brain presumably ingests one word at a time (not sure if it's one exactly, but it can't be much more than one). It is of course true that I have some notion of what I'm going to say before I right the first word, but seemingly so does an LLM.

If it was truly thinking one word at a time, it wouldn't be able to consistently use 'an' vs 'a' correctly, for example.

>we don't restart from scratch for every subsequent word.

LLMs don't restart from scratch for every word, via the attention heads they can look back through the entire context. Otherwise the memory required for inference wouldn't scale with the context length.

Setting aside the silliness of that definition of magic, there's a huge leap between "I can't explain it" and "It can't be explained".

There are plenty of explanations of how LLMs work, by their creators, incidentally.

The thing about LLMs that is revolutionary is just how fast you can find a solution to a problem or expand on answers to a problem even to the point of generating a realistic computational-mathematical model of the problem.

For example, the author brings up Kepler, so let's ask:

> "Please explain in concise terms how Kepler used Tycho Brahe's observational data to come up with Kepler's three laws, on ellipitical orbits sweeping equal areas and the square:cube ratio and so on."

Now I want to see if I can build a computational model of Kepler's Laws:

> "Is there a popular orbital mechanics library for the Python language capable of expressing Kepler's Three Laws in code?"

Okay, now I want a simple model to build in code:

> "How would I go about using poliastro to build a dynamic model of the solar system in silico, starting with just the Sun and the the planets Mercury, Venus, Earth, Mars, Jupiter and Saturn?"

Now trying to use Google Search or anything similar to do that, okay maybe you'd eventually find some forum board or stackoverflow physics discussion of orbital dynamics, but this is an incredibly quick entry point to a complex and obscure subject. Of course, you'd want to use Google Search to check the answers to some degree, maybe see what the real astrophysicists are using to run their models, but there's no doubt that this whole thing is a pretty fundamental game-changer, at least for people who understand its limitations.

P.S. the real question will be if we can build AI systems capabale of generating Kepler's Laws from Tycho Brahe's data, instead of just a predictive model. A similar issue is if these AIs can construct novel mathematical proofs.

Except this doesn't make any sense. Moores law is beginning to fail size has a physical limit in that we can't get much smaller than an atom.

The potential of AI on the other hand is clear. Look at your self. If someone as intelligent as a human can be physically realized then the possibility of building something as intelligent as a human exists by simple logic.

It's easy to see the convoluted argument on your end. Why bet on something that is physically impossible to continue when you can bet on AI that is not only theoretically possible, but walking and talking versions of intelligence are all around us everyday.

The whole premise of this article hinges on the idea that LLMs have fundamental limitations that they clearly don’t have if you’ve looked at lots of gpt4 examples. For example, it can do scientific thinking if you specifically ask it to, and it can reason about totally new situations outside of the training data based on generalizable models of reality it creates to predict training data. If you are certain these limitations must exist based on how LLMs are made, rather than observed capability, it is critical to reevaluate those assumptions.

Moreover, what she defines as scientific thinking is an outdated notion that is no longer widely adopted by scientific researchers, in favor of a more holistic Bayesian process: use intuition to think of something new try, try it, and then update your beliefs with the new data. This is actually more similar to how human brains and LLMs operated before the concept of a scientific method.