Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

321–330 of 641 posts

Re: ML promises to be profoundly weird

#321

Post by charcircuit »

nrabulinski wrote: post #210
drob518 wrote: post #197

Earlier quoted context omitted.

Yes, true. But does that really shift the argument much? An AI is like the most well-read book nerd you’ve ever met. The AI has read everything. They still won’t recite Harry Potter for you at full length and reading what the original author wrote is part of the pleasure.

Does a literal book nerd profit megacorporations when they bring up books to you? While burning through a household worth of energy in the process? Also, I’d like to talk with such book nerd because they’d have opinions on books, potentially if I brought up something I have read we could exchange thoughts about it, they could make recommendations for me based on their complex experiences instead of statistics from Re…

Megacorporations making profit is not some evil that needs to be stopped. The economy is not zero sum.

Re: ML promises to be profoundly weird

#322

Post by camgunz »

simianwords wrote: post #279
camgunz wrote: post #267

Earlier quoted context omitted.

I could quibble with some things, but this is right. I don't have a paid account so I can't ping away at 5.4 or whatever, but, I do have access to frontier models at work, and they hallucinate regularly. Dunno what to do if you don't believe this; good luck I guess.

I agree that they hallucinate sometimes. I agree they bullshit sometimes. But the extent is way overblown. They basically don't bullshit ever under the constraints of 1. 2-3 pages of text context 2. GPT-5.4 thinking I don't think the spirit of the original article (not your comments to be fair) captured this, hence the challenge. I believe we are on the same page here.

> I don't think the spirit of the original article (not your comments to be fair) captured this, hence the challenge. I believe we are on the same page here.

No. GPT-5 has a 40% hallucination rate [0] on SimpleQA [1] without web searching. The SimpleQA questions meet your criteria of "2-3 pages of text content. Unless 5.4 + web searching erases that (I bet it doesn't!) these are bullshit machines.

[0]: https://arxiv.org/pdf/2601.03267

[1]: https://github.com/openai/simple-evals

Re: ML promises to be profoundly weird

#323

Post by [deleted] »

simianwords wrote: post #60
josefritzishere wrote: post #11

I appreciate the directness of calling LLMs "Bullshit machines." This terminology for LLMs is well established in academic circles and is much easier for laypeople to understand than terms like "non-deterministic." I personally don't like the excessive hype on the capabilities of AI. Setting realistic expectations will better drive better product adoption than carpet bombing users with marketing.

If it bullshits so much, you wouldn't have a problem giving me an example of it bullshitting on ChatGPT (paid version)? Lets take any example of a text prompt fitting a few pages - it may be a question in science or math or any domain. Can you get it to bullshit?

[deleted]

Re: ML promises to be profoundly weird

#324

Post by kgeist »

joefourier wrote: post #312

> 2017’s Attention is All You Need was groundbreaking and paved the way for ChatGPT et al. Since then ML researchers have been trying to come up with new architectures, and companies have thrown gazillions of dollars at smart people to play around and see if they can make a better kind of model. However, these more sophisticated architectures don’t seem to perform as well as Throwing More Parameters At The Problem. P…

>Raw parameter counts stopped increasing almost 5 years ago, and modern models rely on sophisticated architectures like mixture-of-experts, multi-head latent attention, hybrid Mamba/Gated linear attention layers, sparse attention for long context lengths, etc.

Agree, I recently updated our office's little AI server to use Qwen 3.5 instead of Qwen 3 and the capability has considerably increased, even though the new model has fewer parameters (32b => 27b)

Yesterday I spent some time investigating it:

- Gated DeltaNet (invented in 2024 I think) in Qwen3.5 saves memory for the KV kache so we can afford larger quants

- larger quants => more accurate

- I updated the inference engine to have TurboQuant's KV rotations (2026) => 8-bit KV cache is more accurate

- smaller KV cache requirements => larger contexts

Before, Qwen3 on this humble infra could not properly function in OpenCode at all (wrong tool calls, generally dumb, small context), now Qwen 3.5 can solve 90% problems I throw at it.

All that thanks to algorithmic/architectural innovations while actually decreasing the parameter count.

Re: ML promises to be profoundly weird

#325

Post by simianwords »

simoncion wrote: post #317
simianwords wrote: post #308

Earlier quoted context omitted.

I specified text just to ignore the voice one because it uses 4o-mini underneath. And its kinda stupid to keep ignoring that and saving face now - reconsider this approach. I believe this is the 5th time I'm asking this: you are not able to produce a _single_ counter example for my challenge? After all this surely I can get a direct acknowledgement here.

> you are not able to produce a _single_ counter example for my challenge? I have. For both your original challenge and your updated one. Consider: 1) AFAICT, there's no way to tell what version of the model was used to produce the output in a ChatGPT share link. 2) You don't appear to believe my assertions that aphyr is almost certainly paying for and using the latest version of the LLMs available, and that he's fai…

Haha ok. So still no example?

The GPT shared link shows a "thought for" which indicates using the latest thinking model. You may try that.

What you can do is this: submit a prompt that clearly makes GPT hallucinate.

You may secretly use a worse model. You may use a system prompt that deliberately gives wrong answers. But I'm going to assume you won't go that far.

We can leave it to the public to decide whether this is a legitimate counter example or not and whether it can really be reproduced. Shall we try that? I'm guessing you won't but worth a shot!

Re: ML promises to be profoundly weird

#326

Post by ModernMech »

lelanthran wrote: post #225
steveklabnik wrote: post #214

Earlier quoted context omitted.

Consent is absolutely important, but that does not mean that every single thing in the entire world requires explicit consent. You did not ask me for consent to use my words in your comment. That does not mean you're a bad person. Free use is an important part of intellectual property law. If it did not exist, the powerful could, for example, stifle public criticism by declaring that they do not consent to you using…

> You did not ask me for consent to use my words in your comment. I am not representing your words as mine. I am not using your words to profit off. I am not making a gain by attributing your words to you. > There has to be lines for cases where requiring attribution is required, and cases where it is not. You are blurring the lines between "using a quote or likeness" and "giving credit to". I am skeptical that you d…

> you don't feel you need any consent from the people you are taking from.

What has been "taken", exactly?

Re: ML promises to be profoundly weird

#328

Post by yumiatlead »

The Industrial Revolution parallel holds up to a point. What it misses: the first industrial revolution required physical coordination — workers, factories, supply chains. The AI revolution requires organizational coordination. Who decides what the agent does, for whom, with whose authority? That governance layer doesn't exist yet, and it's not much a legal question but also an infrastructure question.

Re: ML promises to be profoundly weird

#329

Post by zozbot234 »

janalsncm wrote: post #318
joefourier wrote: post #312

> 2017’s Attention is All You Need was groundbreaking and paved the way for ChatGPT et al. Since then ML researchers have been trying to come up with new architectures, and companies have thrown gazillions of dollars at smart people to play around and see if they can make a better kind of model. However, these more sophisticated architectures don’t seem to perform as well as Throwing More Parameters At The Problem. P…

Yeah I also came here to be one of those People In The Comments the author refers to. Transformers are not magical. They are just a huge improvement over other architectures at the time such as LSTMs and RNNs and even CNNs. They allowed us to throw more and more compute at the problem of next token prediction. And we’ve been riding that horse ever since. Another big advancement that deserves mentioning is “reasoning”…

Transformers' greatest improvement over RNN/LSTM was to enable better parallelization of large-scale training. This is what enabled language models to become "large". But when controlling for overall size, more RNN/LSTM-like approaches seem to be more efficient, as seen e.g. in state space models. The transformer architecture does add some notable capabilities in accounting for long-range dependencies and "needle in a haystack" scenarios, but these are not a silver bullet; they matter in very specific circumstances.

Re: ML promises to be profoundly weird

#330

Post by katatue »

simianwords wrote: post #60
josefritzishere wrote: post #11

I appreciate the directness of calling LLMs "Bullshit machines." This terminology for LLMs is well established in academic circles and is much easier for laypeople to understand than terms like "non-deterministic." I personally don't like the excessive hype on the capabilities of AI. Setting realistic expectations will better drive better product adoption than carpet bombing users with marketing.

If it bullshits so much, you wouldn't have a problem giving me an example of it bullshitting on ChatGPT (paid version)? Lets take any example of a text prompt fitting a few pages - it may be a question in science or math or any domain. Can you get it to bullshit?

I like to let new models write a few lines of Latin poetry - they rarely get the meter right.

I don't have access to paid ChatGPT right now, but here's Opus 4.6 with extra thinking enabled: https://claude.ai/share/6e0e8ef5-06e4-4514-ba7e-299357c1fc55

The initial draft fucks up the meter in lines 3 and 8, the final version gets line 2 wrong ("venit meis") and is somewhat obnoxious with verses 2 and 8 basically repeating each other. The thinking trace is useless and gives us no clue why the model exchanged a bland, but metrically correct first distich for a more interesting, but metrically incorrect one.

In fact, the "careful" examination of its own output completely skips the erroneously modified half-verse in line 2 - now, tell me that's a coincidence and not a sign of bullshitting.

Post reply on HN