Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world

Hmm, it seems that the author takes very clear (and sometimes cynical) positions on some controversial questions. For example, "They don't have the capacity to think through problems logically." is an hotly debated claim, and I think with the advent of reasoning models this has at least become something one should not state in entry level material, which would hopefully reflect common understanding rather than the authors personal opinion in an ongoing discussion.

There are more claims like this about what language models can't do "because they just predict the next token". This line of reasoning, while superficially plausible, holds a lot of assumptions that have been questioned. The heavy lifting here is done by the word "just" - if you can correctly predict the next token in every situation (including novel challenges), does that not require an excellent world model - somehow explicitly reflected in the weights? This is not a settled question but the last few years of LLM success have been completely on the side of those who think that token prediction is quite general.

The material also makes several comparisons to human intelligence, and while it is obvious that humans are different from language models we do not really understand the emergence of all the things that are claimed to be "impossible" for the machine to have in humans (consciousness, morality, etc), it just so happens we are all human so we all agree we have it. Furthermore, it is not clear to me that something can only be called 'intelligent' if it perfectly mimics humans in every way. This is maybe just human bias to our own experience and risks a "submarines can't swim" debate which is really about language.

Many of these philosophical objections have been questioned by people in the field and more importantly by the rapid progress of the models in tasks they were supposed to be incapable of performing according to philosophical objectors. The last few years, every time somebody claims models "can't do X" a new model is released and lo and behold, X is now easy and solved. (If you read a 6 month old paper of impossible benchmarks, expect 75% to be already solved). In fact, benchmark satuation is a problem now. In other words, the goalposts are having trouble keeping up, despite moving at high speed.

I don't think you are doing the general public any service by simply claiming that it is a lot of hype and marketing, these models are really advancing rapidly and nobody really knows where it will end. The philosophical objections seem to be rather weak and are in rapid retreat with every new model, on the other hand the argument in favor of further progress is just "we had progress so far by scaling, if we keep scaling surely we will have more progress" (induction). This is not a strong guarantee of further progress.

The claim that the labs are 'marketing geniuses" for realising language models as chat instead of autocomplete (which they "really" are according to the text - what does that mean?) also seems a bit silly given the obvious utility of the models is already much higher than 'autocomplete'. This seems to be another instance of the common bias that a model that "just" predicts the next token is not allowed to be as succesful as it clearly is in all kinds of tasks.

I don't think a lot of these opinions are particularly well founded and they probably should not be presented in entry level material as if they are facts.

Edit: just to add a positive note, I do think it is extremely useful to educate people on the reliability problem, which is surely going to lead to lots of problems in the wrong hands.

That's an interesting Altman quote on the site. LLMs cannot be compared to electricity and the Internet. People wanted those. LLMs were an impressive parlor trick at first but disappointing later. Many stopped using them altogether.

Now there is a president who fuels the hype, shakes down rich countries for "AI" investments. The Saudi prince who lost money on Twitter is in for the new grift and praises Musk on Tucker Carlson.

The grift-oriented economy might continue with the bailout of Bitcoin whales through the "sovereign wealth fund" scheme.

That is how the "economy" works. No houses will be built and nothing of value will be created.

God says...

bradytrophic indical sacella unlittered surveying mis-tilled brachymetropic oxime masa hypermilitant litas acarotoxic thrust O retranslating awee tetraodont prevoidance cabbala Linker pervertedness absurdest coruscates aminopyrine spitting ledgers Fremontia upthrown simplifiers acomous statutable Ampelopsis

> Being persuasive (i.e., churn out convincing prose) is how LLMs were designed to be.

No. They were designed to churn out accurate prose that accurately reflects their model of reality. They're just imperfect. You're being cynical and emotional to use the term bullshit. And again, it anthropomorphizes the LLM, it implies agency.

We were also told that smart home will make our life easier and save time to have more available for the "important" things in life. But reality is that people just spend more time with their smart home things and are even shouting at their computers even though the computers just do what they were told...