Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

311–320 of 641 posts

Re: ML promises to be profoundly weird

#311

Post by hk__2 »

> This is silly. LLMs have no special metacognitive capacity.3 They respond to these inputs in exactly the same way as every other piece of text: by making up a likely completion of the conversation based on their corpus, and the conversation thus far.

I don’t see how this is silly, because we kind of work the same way. When you do something instinctively and then someone asks you about it, you review the information you (think you) had at the time and from that you produce an explanation.

Re: ML promises to be profoundly weird

#312

Post by joefourier »

> 2017’s Attention is All You Need was groundbreaking and paved the way for ChatGPT et al. Since then ML researchers have been trying to come up with new architectures, and companies have thrown gazillions of dollars at smart people to play around and see if they can make a better kind of model. However, these more sophisticated architectures don’t seem to perform as well as Throwing More Parameters At The Problem. Perhaps this is a variant of the Bitter Lesson.

This is not true and unfortunately this significantly reduced the credibility of this article for me. Raw parameter counts stopped increasing almost 5 years ago, and modern models rely on sophisticated architectures like mixture-of-experts, multi-head latent attention, hybrid Mamba/Gated linear attention layers, sparse attention for long context lengths, etc. Training is also vastly more sophisticated.

The Bitter Lesson is misunderstood. It doesn't say "algorithms are pointless, just throw more compute at the problem", it says that general algorithms that scale with more compute are better than algorithms that try to directly encode human understanding. It says nothing about spending time optimising algorithms to scale better for the same compute, and attention algorithms and LLMs in general have significantly advanced beyond "moar parameters" since the time of Attention is All You Need/GPT2/GPT3.

Re: ML promises to be profoundly weird

#313

Post by sweezyjeezy »

gritspants wrote: post #261
munificent wrote: post #150

There is a whole giant essay I probably need to write at some point, but I can't help but see parallels between today and the Industrial Revolution. Prior to the industrial revolution, the natural world was nearly infinitely abundant. We simply weren't efficient enough to fully exploit it. That meant that it was fine for things like property and the commons to be poorly defined. If all of us can go hunting in the woo…

At what point do we look at 'Industrial Society and its Future' and go from "yeah that'll never happen", "ok some parts of it are happening", to ...? I swear tech folks are the most obtuse people on the planet.

I think it's completely normal. Whenever automation comes knocking, people are inclined to think it's going to flatline conveniently before their job is at risk. LLMs can code now? Cool, they can't code well though can they? Oh they can code pretty well now? Cool, coding was never the hard part of SWE anyway, it's [thing we have no reason to think AI can't beat 99% of humans at at some point], etc

I think SWE as a mainstream profession is much nearer to the end than the beginning, I'm curious and quite scared about what becomes of us.

Re: ML promises to be profoundly weird

#314

Post by roughly »

> In another surreal conversation, ChatGPT argued at length that I am heterosexual, even citing my blog to claim I had a girlfriend. I am, of course, gay as hell, and no girlfriend was mentioned in the post. After a while, we compromised on me being bisexual.

This is a bit of a throwaway in the article, but when people talk about biases encoded in the algorithms, this is what they’re talking about.

Re: ML promises to be profoundly weird

#315

Post by simianwords »

the_snooze wrote: post #266
simianwords wrote: post #143

Earlier quoted context omitted.

I trust you. If it were happening so frequently you may be able to give me a single prompt to get it to bullshit?

I did this in one attempt just now: https://gemini.google.com/share/b4e016be1f69 #8 has an incorrect answer (3 appearances according to Gemini, 2 according to reality https://en.wikipedia.org/wiki/Bowl_championship_series#BCS_a... ) So it works well 95% of the time for literally a trivial use case. Imagine if any other tech tool had that kind of reliability: `ls` displays 95% of your files, your phone successfully se…

Hi! The challenge was ChatGPT but even then it looks like you used the weakest version of Gemini.

Re: ML promises to be profoundly weird

#316

Post by altcognito »

qsera wrote: post #146
rudhdb773b wrote: post #120

Earlier quoted context omitted.

> what actually is happening inside an LLM has nothing to do with conscience or agency What makes you think natural brains are doing something so different from LLMs?

For starters, natural brains have the innate ability to differentiate between things that it knows and things that it have no possibility of knowing...

https://personal.utdallas.edu/~otoole/CGS2301_S09/7_split_br...

See page 53. While it is absolutely more prevelant in LLMs, human brains can also want a story for why their brains do things they are't plugged into.

Re: ML promises to be profoundly weird

#317

Post by simoncion »

simianwords wrote: post #308
simoncion wrote: post #305

Earlier quoted context omitted.

I'm not sure why you're ignoring aphyr's reports. I'm also unsure why you're ignoring my original statement that having the text of the conversation that lead ChatGPT to bullshit is entirely irrelevant, as being unable to repro the report is even worse for ChatGPT than being able to repro would be. shrug

I specified text just to ignore the voice one because it uses 4o-mini underneath. And its kinda stupid to keep ignoring that and saving face now - reconsider this approach. I believe this is the 5th time I'm asking this: you are not able to produce a _single_ counter example for my challenge? After all this surely I can get a direct acknowledgement here.

> you are not able to produce a _single_ counter example for my challenge?

I have. For both your original challenge and your updated one.

Consider:

1) AFAICT, there's no way to tell what version of the model was used to produce the output in a ChatGPT share link.

2) You don't appear to believe my assertions that aphyr is almost certainly paying for and using the latest version of the LLMs available, and that he's faithfully reporting his interactions with the LLMs.

3) Because of #2, I expect that you won't believe me if I report that I've more-or-less reproduced father_phi's results about the cup that's sealed on the top and open on the bottom on the very latest only-available-for-pay ChatGPT model.

3a) You might attempt to check my report, but I'd be shocked if you'd consider a failure to reproduce my results to be a significant strike against ChatGPT. I'd think it's more likely that you'd either call me a liar, or tell me that I must have had some setting wrong somewhere.

3b) Even if you told me to share the ChatGPT chat that proved my assertion, #1 -combined with your demeanor throughout this conversation- tells me that you'd almost certainly claim that I was using an inferior version of the model and was lying to you.

Re: ML promises to be profoundly weird

#318

Post by janalsncm »

joefourier wrote: post #312

> 2017’s Attention is All You Need was groundbreaking and paved the way for ChatGPT et al. Since then ML researchers have been trying to come up with new architectures, and companies have thrown gazillions of dollars at smart people to play around and see if they can make a better kind of model. However, these more sophisticated architectures don’t seem to perform as well as Throwing More Parameters At The Problem. P…

Yeah I also came here to be one of those People In The Comments the author refers to.

Transformers are not magical. They are just a huge improvement over other architectures at the time such as LSTMs and RNNs and even CNNs. They allowed us to throw more and more compute at the problem of next token prediction. And we’ve been riding that horse ever since.

Another big advancement that deserves mentioning is “reasoning” models that have the opportunity to spit out thinking tokens before giving a final answer.

None of this is to say transformers are the most principled approach. But they work.

Re: ML promises to be profoundly weird

#319

Post by [deleted] »

simianwords wrote: post #259
simoncion wrote: post #253

Earlier quoted context omitted.

Neat. So, despite the fact that it looks like you have to pay for ChatGPT Voice mode with video, [0] it doesn't count as an example of it bullshitting on ChatGPT (paid version) That is, father_phi's use of what seems to be a paid version of ChatGPT to have a bullshit-filled conversation that definitely spans less than four pages doesn't count? [0] The page at [1] declares that the video feature is "Available in ChatG…

Lets stick to my challenge please - thinking version, find bullshit. If you can't, thats ok. Do you accept then under the constraints that the thinking version doesn't produce bullshit?

[deleted]

Re: ML promises to be profoundly weird

#320

Post by fwip »

drob518 wrote: post #177
cgh wrote: post #164

Earlier quoted context omitted.

The Vatican Library contains roughly 1.1 million printed books and around 75,000 codices, only a small percentage of which have been digitised.

Which is what percent of the world’s content? 0.000000001% or something similar. It’s nothing in the scheme of things. To put it another way, if we were to digitize that continent and train on it, our AIs would not get noticeably better in any way. It doesn’t move the needle.

1.1 million being 0.000000001% implies a total count of 1e17 books in the world - the real number is closer to 1e8.
Post reply on HN