261–270 of 393 posts
Genuine curiosity here and trying to temper my own selective memory : does it seem like skepticism is growing on the AI front in the news recently?
Elon Bust saying it’s going to replace us gets many eyes on ads even if it’s a dumbass statement.
So just depends on the demographic your news sells ads too.
Earlier quoted context omitted.
Seriously? You think individuals are incapable of reasoning without training first?
Do you think they are? We haven't had a case of individuals without training in like 10 000 years.
"This problem is actually not that easy, the average person couldn't solve it either, especially if the numbers were bigger", "Yet another cherrypicked clickbait study to make LLMs look bad, those people are just scared of being made obsolete", etc.
Earlier quoted context omitted.
Tell me more?
Language is only sequential in the form it is transmitted (verbally). There is no reason that sequential statements are generated sequentially in the brain. Quite the opposite, really, if you consider rules of grammar. I really wish most of the LLM folks just took a few courses in linguistics. It would avoid a lot of noise.
Earlier quoted context omitted.
People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…
The best compression is some form of understanding
The only issue is ambiguity. What can be generated strongly depends on the order of the tokens. A slight variation can change the meaning and the result is worthless. Understanding is the guardrail against meaningless statement and LLMs lack it.
Earlier quoted context omitted.
'It's impossible to plan without an internal monologue.' - Sorry, but I disagree with this. I have no 'internal voice' or monologue - whenever I see a problem, my brain actually and fully models it using images. I believe 25% of the population doesn't have the internal monologue which you're referring to and this has been tested and confirmed. I highly recommend listening to this Lex Friedman podcast episode to get a…
Can you draw a picture of an example of what you see when you think about something?
Earlier quoted context omitted.
I actually did a full write-up on this here fyi: https://photonlines.substack.com/p/intuitive-and-visual-guid... . You can skip most of this and scroll down to the end-section called 'The Mental Model for Understanding LLMs' where I try to map how transformers are able to mimic human thinking. I think that comparing them to auto-associative / auto-regressive networks is actually a really good analogy FYI and I do bel…
We have no idea how human thinking works and no idea what consciousness is. I'm getting so tired of listening to software engineers LARP pseudo neuroscientists with 6th grade level insights. >Of course, the [AI] brain isn’t ‘conscious.’ It doesn’t have any survival instincts which we humans do. Bruh...
Earlier quoted context omitted.
I actually did a full write-up on this here fyi: https://photonlines.substack.com/p/intuitive-and-visual-guid... . You can skip most of this and scroll down to the end-section called 'The Mental Model for Understanding LLMs' where I try to map how transformers are able to mimic human thinking. I think that comparing them to auto-associative / auto-regressive networks is actually a really good analogy FYI and I do bel…
An article this long needs a table of contents IMO.
I just played the game and sent ChatGPT (free, I think 3.5) "Alice has 5 sisters and 3 bothers. How many sister's does Alice's bother have?" The whole thing felt like interacting with your typical support rep who's friendly but otherwise has no common sense and intuition about the thing they're supporting. In other words, it felt like I was interacting with a typical "not so smart but friendly and overconfident" huma…
>"Alice has 5 sisters and 3 bothers. How many sister's does Alice's bother have?" To be fair -- you wrote "bother" twice, and "sister's" with an apostrophe. Technically the question makes no sense like that.
Earlier quoted context omitted.
The best compression is some form of understanding
The best compression relies on understanding. What LLM is is mostly data how humans use words. We understand how to make this data (which is a compression of human text) and use it (generate something). AKA it’s “production rules”, but statistical. The only issue is ambiguity. What can be generated strongly depends on the order of the tokens. A slight variation can change the meaning and the result is worthless. Unde…
Earlier quoted context omitted.
I'm not sure what you mean by 'it will not scale well.' When we humans learn that we make a mistake - we make a note and we hold the correct answer in memory - the next time we're prompted with a similar prompt, we can use our old memories to come up with the correct solution. I just did a simple test for this same exact problem using ChatGPT 3.5: 'Can you reformulate the following problem using Prolog? When you exec…
What if you prompt it "You seem to have accidentally included Alice. The correct answer should be 4"?