Please stop linking to Ted Chiang's article. I like him as an imaginative writer, but his article is just wrong and gives readers incorrect intuitions. His claim that GPT models are not able to learn decimal addition has been known to be false for years and you can verify yourself that GPT-4 can do it.
ChatGPT as a Calculator for Words
31–40 of 81 posts
Re: ChatGPT as a Calculator for Words
#32The author does a good job of pointing out what may be the strongest skills of LLMs but the claim they aren't useful as a search engine didn't ring particularly true. For many questions I have ChatGPT is the best tool to use because I know the topics I'm asking about are mentioned hundreds of times in the web, and the LLM can distill down at knowledge to the specifics I'm asking about. If you treat it as a friend who…
It would be nice if it wasn't making up nearly all the links and sources I ask of it.
Re: ChatGPT as a Calculator for Words
#33The most telling thing about the state-of-the-art currently is that somewhere between all the marketecture in no-code and low-code, we can essentially create a recursive crawler on-the-fly on par with a large cloud provider (because it is) and ask it to do recursive crawls that at least begin to approach a sensemaking machine, i.e. it can provide consensus for subjective truth agreed upon by experts above and beyond simple objective truth. To me, that's the great innovation that truly embraces and extends average intelligence by providing a prosthetic device for the brain to make inferences that would have only been approachable by high functioning individuals previously, i.e. the kind of argument about consciousness that you see from low latency inhibition researchers like Peterson. What any individual does with this is really where prompt engineering becomes the human-computer agent collaboration that should be our default mode in computing - a kind of tortoise wins the race story where we lost our minds in the race to interaction via javascript. It's not terribly interesting to watch a computer type. There's a place for batch mode (queuing, etc) , if the tools built up around it handle long running job management well. Sadly, that seems rarer to me now than 30 years ago.
Re: ChatGPT as a Calculator for Words
#34As a long time LLM enjoyer I just want to mention https://generative.ink/posts/simulators/ as I think it's by far the most insightful take on the GPT LLMs even though it was from before ChatGPT. It's better than blurry jpeg and stochastic parrot etc.
Yup, that's a classic article.
My favorite is Chalmer's engine bit:
> What pops out of self-supervised predictive training is noticeably not a classical agent. Shortly after GPT-3’s release, David Chalmers lucidly observed that the policy’s relation to agents is like that of a “chameleon” or “engine”:
>> GPT-3 does not look much like an agent. It does not seem to have goals or preferences beyond completing text, for example. It is more like a chameleon that can take the shape of many different agents. Or perhaps it is an engine that can be used under the hood to drive many agents. But it is then perhaps these systems that we should assess for agency, consciousness, and so on.6
Re: ChatGPT as a Calculator for Words
#35> Language models don’t—if you run the same prompt through a LLM several times you’ll get a slightly different reply every time.
You can get deterministic output (on a given machine) by setting temperature=0. The Chatgpt interface doesn't let you do that, but the playground API does.
Re: ChatGPT as a Calculator for Words
#36> Language models don’t—if you run the same prompt through a LLM several times you’ll get a slightly different reply every time. You can get deterministic output (on a given machine) by setting temperature=0. The Chatgpt interface doesn't let you do that, but the playground API does.
Also if you specify the seed you get deterministic output, assuming that the interface allows to specify a seed.
Re: ChatGPT as a Calculator for Words
#37Earlier quoted context omitted.
Is this a result of the machine-instead-of-human-reviewer/editor difference, or because of a sense that the writing is less a "pure" output of a singular author?
Well I think it's like, you come to enjoy an author's particular style of writing, and if people are just going to use ChatGPT to write things for them, then they're not going to develop any style. Everyone might even end up all sounding the same. With a calculator this is a feature. We want computations to be the same after all. Everyone should be able to get the same results when they enter the same numbers in. But…
Part of the appeal of ChatGPT compared to traditional writing assistant/grammar checkers is that you can tell it to write in different personas, although IME it's pretty spotty at this still. Lots of conversations that turn bland and repetitive quickly if you don't get lucky.
I'm not too worried about "default GPT style" becoming common, though, because I think it's more likely to be used by the people who have no style beyond "what I see on TV and in my family." Raising the floor, basically.
Anyone who wants their writing to stand out will still have to differentiate themselves. To put it another way: you're gonna be able to recognize the lazy users pretty quickly cause they're gonna have "GPT voice."
Re: ChatGPT as a Calculator for Words
#38As a long time LLM enjoyer I just want to mention https://generative.ink/posts/simulators/ as I think it's by far the most insightful take on the GPT LLMs even though it was from before ChatGPT. It's better than blurry jpeg and stochastic parrot etc.
One thing I've been impressed by recently is ChatGPT's ability to explain epigrams.
How can it explain them effectively if it's just a "stochastic parrot" or a "blurry jpeg"? Neither fits what it does.
Don't such effective explanations require some sort of understanding?
Below is an example explanation, and there are more here: [1]
---
Explain "In software systems it is often the early bird that makes the worm."
This statement refers to the importance of early planning and implementation in software systems. In software development, the early stages of the project, such as requirements gathering, design, and implementation, are crucial for the success of the project.
If the early stages of the project are done well, it can set the foundation for the rest of the development process, making it easier to build and maintain the software system. On the other hand, if these stages are not done well, it can lead to delays, rework, and budget overruns.
The phrase "the early bird that makes the worm" is a reference to the proverb "the early bird catches the worm", which means that those who act early and decisively will have an advantage over others.
Similarly, in software development, if the development team acts early and puts effort into the early stages of the project, they can gain an advantage over competitors, reducing the risk of project failure, and increasing the chances of success.
In summary, the statement "In software systems, it is often the early bird that makes the worm" emphasizes the importance of early planning and implementation in software development, which can set the foundation for the rest of the development process and increase the chances of success.
[1] - https://diaphora.blogspot.com/2023/04/chatgpt-explains-epigr...
Re: ChatGPT as a Calculator for Words
#39As a long time LLM enjoyer I just want to mention https://generative.ink/posts/simulators/ as I think it's by far the most insightful take on the GPT LLMs even though it was from before ChatGPT. It's better than blurry jpeg and stochastic parrot etc.
One thing I've been impressed by recently is ChatGPT's ability to explain epigrams. How can it explain them effectively if it's just a "stochastic parrot" or a "blurry jpeg"? Neither fits what it does. Don't such effective explanations require some sort of understanding? Below is an example explanation, and there are more here: [1] --- Explain "In software systems it is often the early bird that makes the worm." This…
That's not even a good explanation for that epigram and the output is similar to many written pieces of content on the internet, which ChatGPT almost surely borrows from via CommonCrawl.
Re: ChatGPT as a Calculator for Words
#40I've seen this take a lot, and I find it frustrating as it flies in the face of the information theory underpinning how large neural networks learn information. This is more than just a fancy zip file of Markov sequences. Someone has got to put a stop to this silly line of reasoning, I'm not sure why more people familiar with the math of deep learning aren't doing their best to dispel this particular belief (which pe…
Can you explain more ? I know the basics of deep learning and I found the article accurate.
As an amatuer I think Markov chains are explicitly a crude frequency association whereas what exactly a neural network is storing to predict the next token involves stored representations in neural weights which can be far more nuanced.