141–150 of 518 posts
Earlier quoted context omitted.
Ludwig Wittgenstein had same idea. but finally he found human experience is more then our language.
Could you elaborate please? For those of us not familiar with Wittgenstein's work, could you link to sources depicting before and after his views changed, preferably with summaries.
Summary of after: https://en.wikipedia.org/wiki/Philosophical_Investigations
Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…
I think not very far in the future we are gonna look at the belief that there is something extraordinary about human thought the way we think about geocentrism. That we privileged the human brain simply because it’s ours, because I suspect human thought will end up being not much more than something like a pattern matching mechanism.
The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…
Finally I'm tired of people saying it's just a probabilistic word generator and downplaying everything as if they know. If you said something along these lines before... then these papers show that you're not fully grasping the situation here. There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical w…
Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room.
Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyrics!
Earlier quoted context omitted.
I didn't say it's not a probabilistic word generator. I'll quote more of what I wrote: >There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical word jumbles. So CLEARLY because I said there are different angles of interpretation I'm implying that from one of these angles we can interpret it as a stat…
I mean, that is the fundamental architecture of the model and how it's explicitly designed: given an input embedding and output embedding, the model is trained over a massive corpus of text to optimize the probability for generating the next token in the sequence. Any other interpretation is secondary to that, no matter how impressive the results are.
Evolutionary biology strives to make us logical creatures to fulfill the singular goal of passing on genetic material. Your sentience and your humanity is a side effect of this singular goal.
So what dominates the description of who YOU are? Human or vessel for genetic material?
I'll say that YOU are human and therefore more then just a vessel for ferreting your genetic material into the future... just like how I'll go with the fact that LLMs are more then just statistical word generators.
Earlier quoted context omitted.
In my experience, ChatGPT is unsuitable even for tic tac toe. When I tried it, it hallucinated completely incorrect board positions (e.g. forgot about moves already made or wanted to continue playing after I had won).
You (or running software you interface with ChatGPT) needs to maintain the state of the game between moves. LLMs aren’t computers but they can interact with computers. LLM(InitialInstructions)->Computer(CodeWrittenByLLM)->LLM(InstructionsOutputByComputer)->LoopUntilWin
The easiest way for ChatGPT to generate good output is to plain understand it. Given the vast amount of input data fed into it, it has no choice, but to start reducing the input into fundamental rules which is basically what understanding is. Understanding is a form of compression. More efficient for a neural network to understand a concept than memorize permutations. Same with statistics and markov chains, people fo…
I used to believe this, but then I saw the demo of ChatGPT attempting to play chess. The rules of chess are much simpler than any structure of the real world we might hope for it to understand—and yet it failed to learn the rules of chess. Based on this, I am pretty sure it has not learned any meaningful structure.
Earlier quoted context omitted.
Doesn't the article say... exactly that? The basic concept of ChatGPT is at some level rather simple. Start from a huge sample of human-created text from the web, books, etc. Then train a neural net to generate text that’s “like this”. And in particular, make it able to start from a “prompt” and then continue with text that’s “like what it’s been trained with”. Just because there are emergent behaviors doesn't mean i…
I didn't say it's not a probabilistic word generator. I'll quote more of what I wrote: >There are clearly different angles of interpreting what these models are actually doing but people are stubbornly refusing to believe it's anything more then just statistical word jumbles. So CLEARLY because I said there are different angles of interpretation I'm implying that from one of these angles we can interpret it as a stat…
It sounds like you just don't like viewing it as simply "a probabilistic word generator", as that takes the magic out of it. And yeah, it's not magic, but it is quite useful.
I’d like it on iPadOS because that’s where I like to read and write. I tried reader mode, but it lost a lot of the images.
Any suggestions?
Edit: I was able to get a good PDF using the OneNote web clipper on my desktop.
Off topic: I’m looking for a good web clipper to grab just the article and related images from a page like this and save it (as html or a pdf) for me to read later. I’d like it on iPadOS because that’s where I like to read and write. I tried reader mode, but it lost a lot of the images. Any suggestions? Edit: I was able to get a good PDF using the OneNote web clipper on my desktop.