Off topic: I’m looking for a good web clipper to grab just the article and related images from a page like this and save it (as html or a pdf) for me to read later. I’d like it on iPadOS because that’s where I like to read and write. I tried reader mode, but it lost a lot of the images. Any suggestions? Edit: I was able to get a good PDF using the OneNote web clipper on my desktop.
https://www.google.com/search?q=web+to+pdf
What is ChatGPT doing and why does it work?
161–170 of 518 posts
Re: What is ChatGPT doing and why does it work?
#162Earlier quoted context omitted.
> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyri…
It's clearly early technology, so it's not perfect. But what it is able to get right is clear proof it's more then what you think: https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning and middle doesn't show off anything too impressive. It's the very end where chatGPT displays a sort of self awareness. Also here's a scientific paper showing that LLMs are more then a chinese room…
Re: What is ChatGPT doing and why does it work?
#163Earlier quoted context omitted.
> 'with enough data its easier to model than we expected' > a self-similar process is ultimately responsible for granting us our own intelligence In my view, intelligence essentially resides within language, specifically in the corpus of language. Both humans and AIs can be effectively colonized by language, as there are innumerable concepts and observations that are transmitted from one mind to another, and now even…
Yann LeCun often argues that animals like cats and dogs are substantially more intelligent than current LLMs [0] and I'd have to agree. I don't see how/why to consider practical knowledge as only constituting a tiny fraction of our overall intelligence. Either way, it's not clear if the GPT-* models will someday produce emergent common sense or if they're going down an entirely wrong path. [0] https://twitter.com/yle…
Someone objected that a cat can't write a Python program, and LeCun points out that "Regurgitating Python code does not require any understanding of a complex world." No, but a) interpreting the prompt does require understanding, and good luck finding a dog or cat who will offer any response at all to a request for a Python program; and b) it's hardly "regurgitating" if the output never existed anywhere in the training data.
TL,DR: his FoMO is showing.
Re: What is ChatGPT doing and why does it work?
#164Off topic: I’m looking for a good web clipper to grab just the article and related images from a page like this and save it (as html or a pdf) for me to read later. I’d like it on iPadOS because that’s where I like to read and write. I tried reader mode, but it lost a lot of the images. Any suggestions? Edit: I was able to get a good PDF using the OneNote web clipper on my desktop.
Re: What is ChatGPT doing and why does it work?
#165Earlier quoted context omitted.
How is this significant? I mean you can view a moving car as stationary, and reality is the thing that moves around it, or you can view reality as stationary, but the car moves through it. Or you can make analogies between electricity and water. (Frankly, I like viewing neural nets as a numerical method for doing nonlinear multivariate regression, and ChatGPT is nothing but a resulting equation(s) of such). In the en…
Well you're just a jumble of electrons and protons interacting with each other. That's a 100% true interpretation is it not? It's also a view that misses the point that you are a molecular intelligence made of DNA that continually mutates and reconstructs it's physical form with generational copies to increase fitness in an ever changing environment. But that viewpoint also misses the point that you're a human with w…
Your last paragraph lacks proof, and is a reflection of how you feel and want to view it - as something more than a statistical word generator. That's fine, but people with graduate level math/statistics education know that math/statistics is capable of doing everything ChatGPT does (and even more). To me, it sounds like you're the fearful one.
Re: What is ChatGPT doing and why does it work?
#166Off topic: I’m looking for a good web clipper to grab just the article and related images from a page like this and save it (as html or a pdf) for me to read later. I’d like it on iPadOS because that’s where I like to read and write. I tried reader mode, but it lost a lot of the images. Any suggestions? Edit: I was able to get a good PDF using the OneNote web clipper on my desktop.
I am reading the original webpage on Google Chrome on Win11 and I have the Microsoft Print to PDF driver installed. A simple print to PDF through this virtual printer seems to give a decent PDF file.
Re: What is ChatGPT doing and why does it work?
#167Earlier quoted context omitted.
You (or running software you interface with ChatGPT) needs to maintain the state of the game between moves. LLMs aren’t computers but they can interact with computers. LLM(InitialInstructions)->Computer(CodeWrittenByLLM)->LLM(InstructionsOutputByComputer)->LoopUntilWin
In my experiments that doesn’t help. ChatGPT failed to play any valid move given a position, let alone a good one. The point is that language is probably too “linear” to represent what’s going on during a game. Pieces on the board have complex relationships (consider a pinned piece, for example) so autoregressive decoding is simply not enough.
The current board state is:
board = [['', '', ''],
['', '', ''],
['', '', '']];
write a javascript function called bestMove(board) that predicts the best tic-tac-toe move to make given a board. use that function to update the board state and return the board state in JSON form.
The response will have a bunch of functions like function bestMove(board) {
function getEmptySpaces(board) {
function predictBestMove(board, player) {
function minimax(board, isMaximizing) {
function checkWinner(board) {
...
These functions should work together to determine the best move to make in a game of Tic-Tac-Toe, using the minimax algorithm to evaluate each possible move and choosing the one with the highest score.
Then eval and execute the bestMove function, passing in the initial board state, returning the updated board state. Then the human player makes a move.Then another prompt:
Try something like:
The current board state is:
board = [['X', '', ''],
['', 'O', ''],
['', '', '']];
assume there is a function called bestMove(board) and checkWinner(board) that predicts the best tic-tac-toe move to make given a board. use those functions to update the board state and check the winner and return the board state and current winner in JSON form.
etc...Using my little engine, I get this solution:
question: "Answering as [rowInt, colInt], writing custom predictBestMove, getEmptySpaces, minimax and checkWinner functions implemented in the thunk, what is the best tic-tac-toe move for player X on this board: [['X', '_', 'X'], ['_', '_', '_'], ['_', '_', '_']]?",
answer: [ 0, 1 ],
https://gist.github.com/williamcotton/e6bdcca0a96a6e7bf5d2fe...Re: What is ChatGPT doing and why does it work?
#168Earlier quoted context omitted.
> 'with enough data its easier to model than we expected' > a self-similar process is ultimately responsible for granting us our own intelligence In my view, intelligence essentially resides within language, specifically in the corpus of language. Both humans and AIs can be effectively colonized by language, as there are innumerable concepts and observations that are transmitted from one mind to another, and now even…
Intelligence exists without language, language is only a way to describe the world around us and transfer information. We personally can't experience intelligence without language because you know language all your life and can't remember a moment in which you didn't. But there were humans in history that didn't knew any language and they were intelligent, there are animals that do not know any language and are intel…
Re: What is ChatGPT doing and why does it work?
#169Earlier quoted context omitted.
> deep understanding of self and awareness Come on... you're making it sound like the thing is sentient. It's impressive but it's still a Chinese Room. Although, for searching factual information it still failed me.. I wanted to find a particular song - maybe from Massive Attack or a similar style - with a phrase in the lyrics, I asked Chatty, and it kept delivering answers where the phrase did not appear in the lyri…
It's clearly early technology, so it's not perfect. But what it is able to get right is clear proof it's more then what you think: https://www.engraved.blog/building-a-virtual-machine-inside/ Read to the end. The beginning and middle doesn't show off anything too impressive. It's the very end where chatGPT displays a sort of self awareness. Also here's a scientific paper showing that LLMs are more then a chinese room…
I guess it's a Chinese Room, that when you ask about Chinese Rooms, can tell you what those things are. I almost said the word "aware" there, but the person in the Chinese Room, while composing the answer to "What is a Chinese Room?" isn't aware that "Wait a minute, I'm in a Chinese Room!", because s/he can arrange Chinese sentences, but s/he just knows what characters go with what, s/he doesn't know the meaning or emotion behind those words.
And if you ask him/her "Are you in a Chinese room?", they can answer according to the rules given to them (the Chinese word for "Yes", for example), but there surely isn't a contemplation about e.g. "Why am I in this room?".
If you ask ChatGPT about the Turkish earthquake, it can give you facts and figures, but it won't feel sad about the deaths. It can say it feels sad, but that'd be just empty words.
Re: What is ChatGPT doing and why does it work?
#170The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…
Neural Networks and the Chomsky Hierarchy [Deep Mind 2022] https://arxiv.org/abs/2207.02098