231–240 of 244 posts
Earlier quoted context omitted.
So the consensus is that consciousness is computable by a Turing machine?
The consensus is that if it's not, it's not because of the reasons Penrose gives.
Earlier quoted context omitted.
Here's an example of it failing to understand some very simple Python code that any human with basic programming knowledge would not mess up: ------ Me: What is going to be the output of this code when `x = 0` and when `x = 900`: ```python x = 0 for i in range(100): if x > 50: x +=1 else: x += 5 if x == 90: print("X is not 90") else: pass ``` GPT-4: When x = 0, the code will execute as follows: The initial value of x…
GPT-4 (as well as all GPTs before it has a limitation): it has to produce an output in a single pass. It cannot pause and think, it cannot backtrack. So yes, it makes weird mistakes sometimes. A human programmer will need to look at code, then think a bit, then look at it again, etc. You can put programmers into a similar situation: try reading code aloud instead of showing it to them on screen. If they can't answer…
https://ar5iv.labs.arxiv.org/html/1706.03762
"Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences ...
In this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output."
Beyond that, also note that LLMs are probabilistic machines. Output spat out can vary and there are a handful of knobs (such as temperature) to modulate that output.
Finally, I'm pretty sure we (or the workers in the field more like it /g) don't have a firm grasp on why certain failure modes occur. Likely this is due to the fact that we (they) also don't really have a good grasp on how the damn thing actually works its 'magic'.
What is clear is that a significant subset of our semantic universe is embedded in symbols and their usage by us and this subset is somehow encoded in neural nets. This captured subset in LLMs is what drives their uncanny generative abilities. What is missing is precisely what would make it plausibly intelligent, plausibly a reasoning agent operating in a coherent semantic context.
There are some who claim our minds are just like LLMs. Some of us who pay attention to our minds sometimes catch it making nonsensical noises and correct it. (As you age you begin to notice these things..) So it is interesting to this sentient (who makes claims to being) that my mind is just like my body, it is aging, certain parts are degraded, etc., but my 'whateveritis' that is me, my self, is as timeless as ever, and seems to be a spectator of the aging mechanism ..
Earlier quoted context omitted.
Here's an example of it failing to understand some very simple Python code that any human with basic programming knowledge would not mess up: ------ Me: What is going to be the output of this code when `x = 0` and when `x = 900`: ```python x = 0 for i in range(100): if x > 50: x +=1 else: x += 5 if x == 90: print("X is not 90") else: pass ``` GPT-4: When x = 0, the code will execute as follows: The initial value of x…
GPT-4 (as well as all GPTs before it has a limitation): it has to produce an output in a single pass. It cannot pause and think, it cannot backtrack. So yes, it makes weird mistakes sometimes. A human programmer will need to look at code, then think a bit, then look at it again, etc. You can put programmers into a similar situation: try reading code aloud instead of showing it to them on screen. If they can't answer…
Earlier quoted context omitted.
It's wrong. A decoder only transformer performs a (possibly random) operation on a state from the state space {tokens}^CtxWindow, where the distribution of the new state depends entirely on the previous state. It is a Markov Chain with a special structure: The new state is deterministically equal to the old state shifted by one, with only the last token being newly generated.
Then by that reasoning everything in the physical world is a Markov chain, right? That is like saying that any deterministic process in time is a Markov chain. A tennis ball in flight is a Markov chain since the state at t is a function of the state at t-1. You have missed the point about the Attention Mechanism in GPT. That is not a Markov chain by definition.
Well I guess maybe it's true that you can turn any stochastic process into a Markov Chain by changing the state space somehow (for example the states could be sample trajectories up to some finite time T). And while this is true it may be not very insightful.
But I personally think that to understand LLMs it is much better to think of the whole context window as a state rather than the individual states. If you modelled a simple register-instruction computer as a stochatic process, would you take the states to be (address last symbol written, last symbol written)? It makes much more sense to take the whole memory as a state. Similarly a transformer operates on its memory, the context window, so that should be seen as the state. This makes it clear that seeing it as just a stochastic parrot is misleading, as its all about conditioning the distribution of the next token via prompt engineering the previous tokens. And it is nevertheless a Markov chain with this state space.
Earlier quoted context omitted.
It is a Markov Chain on the state space {Tokens}^CtxWindow.
I don't think that's clear at all. https://arxiv.org/abs/2212.10559 shows a LLM is doing gradient descent on the context window at inference time. If it's learning relationships between concepts at runtime based on information in the context window then it seems about as useful to say it is a Markov chain as it is to say that a human is a Markov chain. Perhaps we are, but the "current state" is unmeasurably complex.
Earlier quoted context omitted.
Then by that reasoning everything in the physical world is a Markov chain, right? That is like saying that any deterministic process in time is a Markov chain. A tennis ball in flight is a Markov chain since the state at t is a function of the state at t-1. You have missed the point about the Attention Mechanism in GPT. That is not a Markov chain by definition.
>Then by that reasoning everything in the physical world is a Markov chain, right? Well I guess maybe it's true that you can turn any stochastic process into a Markov Chain by changing the state space somehow (for example the states could be sample trajectories up to some finite time T). And while this is true it may be not very insightful. But I personally think that to understand LLMs it is much better to think of…
Earlier quoted context omitted.
>Then by that reasoning everything in the physical world is a Markov chain, right? Well I guess maybe it's true that you can turn any stochastic process into a Markov Chain by changing the state space somehow (for example the states could be sample trajectories up to some finite time T). And while this is true it may be not very insightful. But I personally think that to understand LLMs it is much better to think of…
So basically you're saying it's just an algorithm running on a computer? Yes I agree with that.
You know, I think that some people (I see on twitter, probably not you) have a wrong intuition about artificial intelligence. They see models which are fundamentally stochastic as incapable of really ever being truly intelligent. It's "just statistics" or just a "stochastic parrot" and it just learns probabilities instead of real meaning. Perhaps they think that since there is always randomness involved, you can not have the kind of deterministic thought process that we feel we have. The worst offender is then considered to be the old school Markov chain.
I obviously think this is wrong and that's why I like to emphasize that transformers are best interpreted as Markov Chains on a larger state space, and this does actually explain their computational behavior.
Earlier quoted context omitted.
Well it’s not quite that simple. Brains use spiking neural networks, not the kind used typically in artificial neural networks like those used by LLMs. The “weights” can be changed over time, new connections and even new neurons formed. And the number of connections (“weights”) is about 500-1000x more in our brain than GPT-3. The connection topography is a lot different. But ultimately, our brains are still just made…
Do you consider that every neuron in the brain has unique DNA and ancestorship ?
https://www.scientificamerican.com/article/scientists-surpri...
https://www.science.org/doi/10.1126/science.aab1785 - Somatic mutation in single human neurons tracks developmental and transcriptional history
(good luck simulating that)
Earlier quoted context omitted.
GPT-4 (as well as all GPTs before it has a limitation): it has to produce an output in a single pass. It cannot pause and think, it cannot backtrack. So yes, it makes weird mistakes sometimes. A human programmer will need to look at code, then think a bit, then look at it again, etc. You can put programmers into a similar situation: try reading code aloud instead of showing it to them on screen. If they can't answer…
GPT is a transformer model. Transformers use the attention mechanims. The mechanism is entirely concerned with retaining semantic context and semantic "global dependencies" spanning the entire input and output. https://ar5iv.labs.arxiv.org/html/1706.03762 "Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies witho…
This is not quite true: GPT, specifically, is auto-regressive. It computes things only looking back, not forward.
Given that each token has only a fixed computing budget, it is likely that GPT precomputes information which will be relevant to later tokens, to be routed via attention.
In fact, this effect was demonstrated in practice: e.g. in a prompt like "Question: Where is the Eiffel tower located? Answer: " people found that information about "Paris" is routed from tokens "Eiffel tower", i.e. this associative memory was looked up earlier than it was needed.
So I was answering from that perspective: it can do better if it knows what to pre-compute.