One Head, Two Brains: The origins of split-brain research (2015)

I think this is what severance is about.

The confidence seems to be an artifact of fine tuning. The first instruction trained models were given data sets with answers to questions but generally omitted non answers to things the model didn't know.

Later research showed that models know that they don't know certain pieces of information, but the fine tuning constraint of providing answers did not give them the ability to express that they didn't know.

Asking the model questions against known information can produce a correct/incorrect map detailing a sample of facts that the model knows and does not know. Fine tuning a model to say "I don't know" in response to the those questions where it was incorrect can allow it to generalise the concept to its internal concept of unknown.

It is good to keep in mind that the models we have been playing with are just the first ones to appear. GPT 3.5 is like the Atari 2600. You can get it provide a limited experience for what you want and its cool that you can do it at all, but it is fundamentally limited and far from an ideal solution. I see the current proliferation of models to be like the Cambrian explosion of early 8 bit home computers. Exciting and interesting technology which can be used for real world purposes, but you still have to operate with the knowledge of the limitations forefront in your mind and tailor tasks to allow them to perform the bits they are good at. I have no-idea of the timeframe, but there is plenty more to come. There have been a lot of advances revealed in papers. A huge number of those advances have not yet coalesced into shipping models. When models cost millions to train you want to be using a set of enhancements that play nicely together. Some features will be mutually exclusive. By the time you have analysed the options to find an optimal combination, a whole lot of new papers will be suggesting more options.

We have not yet got the thing for AI that Unix was for computers. We are just now exposing people to the problems that drives the need to create such a thing.

Different LLMs can also come to the same conclusions, that is, predict the same strings of tokens (in the meaning).

Sure, they're a far way from the capacity of humans in terms of short term training. But there is literally nothing that indicates they can't "think" (understand, reason, abstract, whatever word you wanna put in italics) because nobody can explain what it really means, because: it's all just predicting. Text happens to be super useful for us to evaluate certain aspects of predicting.

>Miller’s study uses a test called the “trait-judgment task”: A trait like happy or sad flashes on a screen, and research subjects indicate whether the trait describes them. Miller has slightly modified this task for his split-brain patients—in his experiments, he flashes the trait on a screen straight in front of the subject’s gaze, so that both the left and right hemispheres process the information. Then, he quickly flashes the words “me” and “not me” to one side of the subject’s gaze—so that they’re processed only by one hemisphere—and the subject is instructed to point at the trait on the screen when Miller flashes the appropriate descriptor.

Seems to me (not a neuroscientist) like there's a flaw in that experiment: how would the right hemisphere understand the meaning of the words, if language is only processed by the left? I also recall reading that the more "primitive" parts of our brains don't have a concept of negation.

But maybe they have been considering this and it's no issue?

I always thought it was interesting that the human brain grew relatively quickly in evolutionary history. 3 million years ago, our ancestors had a 400 cc brain. 2.5 million years later, it was 1,400 ccs--more than 3 times larger.

That implies to me that a larger brain immediately benefited our ancestors. That is, going from 400 to 410 ccs had evolutionary advantage and so did 410 to 420, etc.

That implies that once the brain architecture was set, you could increase intelligence through scale.

I bet there are some parallels to current AI there.