Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ML promises to be profoundly weird

aphyr.com

Post reply on HN
Print view View original

231–240 of 641 posts

Re: ML promises to be profoundly weird

#231

Post by nyeah »

ajross wrote: post #128
nyeah wrote: post #116

Earlier quoted context omitted.

Good point. We choose certain tests to perform. We choose certain test results to pay attention to. We don't just keep chatting about (reviewing) the code. We do something else. If lies are all we have, then how is this behavior possible?

LLMs can write and run tests though. You're cherry picking my little bit of wordsmithing. Obviously we aren't always wrong. I'm saying that our thought processes stem from hallucinatory connections and are routinely wrong on first cut, just like those of an LLM. Actually I'm going farther than that and saying that the first cut token stream out of an AI is significantly more reliable than our personal thoughts. Certa…

I don't think the complaint about cherry picking is quite fair. Most of your original comment consists of claims that we're bullshit machines, our internal dialog is almost 100% fantasy, we're hallucinating, etc. Those claims may be true. But I'm not carefully like curating them out of nowhere.

Re: ML promises to be profoundly weird

#232

Post by simianwords »

simoncion wrote: post #227
simoncion wrote: post #166

Earlier quoted context omitted.

> If it bullshits so much, you wouldn't have a problem giving me an example of it bullshitting on ChatGPT (paid version)? There's an entire paragraph in the essay about apyhr's direct experience with ChatGPT failures and sustained bullshitting that we'd never expect from a moderately-skilled human who possesses at least two functioning braincells. That paragraph begins "I have recently argued for forty-five minutes w…

There's also this seven-week-old example [0] (linked in the essay) of ChatGPT very confidently recommending a asinine course of action because it was unable to understand what the hell it was being told. Listening to the audio is not required, as there's a reasonably accurate on-screen transcript, but it is valuable to listen to just how very hard they've worked to make this tool sound both confident and capable, eve…

This is classic case of not using the proper version. Use the thinking version gpt5.4 (text) and tell me if it bullshits.

Surely you must be able to find at least one example no?

Re: ML promises to be profoundly weird

#233

Post by simianwords »

perching_aix wrote: post #230
simianwords wrote: post #66

Earlier quoted context omitted.

[flagged]

My personal red flag for this is the scare quoting of AI, and the super try-hard categorization work that people perform to try and discredit LLMs. It takes approximately 1 min to find out that machine learning is a subfield of artificial intelligence, both having existed for about half a century now. This basic historical fact is also taught on AI 101 courses across the globe for compsci students. Yet here we are, p…

I think it’s just anxiety because to internalise that it is actually so good is a bit hard for some

Re: ML promises to be profoundly weird

#234

Post by n4r9 »

NiloCK wrote: post #89
n4r9 wrote: post #40

Earlier quoted context omitted.

The concern for me about LLMs confabulating is not that humans don't do it. It's that the massive scale at which LLMs will inevitably be deployed makes even the smallest confabulation extremely risky.

I don't understand this. Many small errors distributed across a large deployment sounds a lot like normal mode of error prone humans / cogs / whatevers distributed over a wide deployment.

Let's say a given B2B system deployment typically requires 100 custom behaviours/scripts and 3 years worth of effort. A team of ten people can execute such a deployment in 3-4 months. The team has the capacity to fix up issues caused by small human errors as they arise, since they show up roughly once a week.

With the advent of LLMs, a new deployment now takes 3 days. Consequently, errors requiring human attention crop up several times a day.

Re: ML promises to be profoundly weird

#235

Post by htrp »

drob518 wrote: post #149
htrp wrote: post #114

Earlier quoted context omitted.

We pay people to create more high quality tokens (mercor, turing) which are then fed into data generating processes (synthetic data) to create even more tokens to train on

But does that really help, or do you get distortion? The frequency distribution of human generated content moves slowly over time as new subjects are discussed. What frequency distribution do those “data generating processes” use? And at root, aren’t those “data generating processes” basically just another LLM (I.e., generating tokens according to a probability distribution)? Thus, aren’t we just sort of feeding AI s…

That's the problem in a nutshell. There is an art to how you generate the synthdata so that you don't get crappy trained models (especially when mistakes cost XX million dollars).

It's also theoretically why facebook paid 14bn for alex wang and scale ai

Re: ML promises to be profoundly weird

#236

Post by lamasery »

> People keep asking LLMs to explain their own behavior. “Why did you delete that file,” you might ask Claude. Or, “ChatGPT, tell me about your programming.”

Oh man, every business-side person in my company insists on reporting all the way to the UI a "confidence score" that the LLM generates about its own output and I've seen enough to know not to get between an MBA and some metric they've decided they really want even if I'm pretty sure the metric is meaningless nonsense, but... I'm pretty sure those are meaningless nonsense.

Re: ML promises to be profoundly weird

#237

Post by simoncion »

simianwords wrote: post #232
simoncion wrote: post #227

Earlier quoted context omitted.

There's also this seven-week-old example [0] (linked in the essay) of ChatGPT very confidently recommending a asinine course of action because it was unable to understand what the hell it was being told. Listening to the audio is not required, as there's a reasonably accurate on-screen transcript, but it is valuable to listen to just how very hard they've worked to make this tool sound both confident and capable, eve…

This is classic case of not using the proper version. Use the thinking version gpt5.4 (text) and tell me if it bullshits. Surely you must be able to find at least one example no?

To be clear, is your assertion that apyhr was also not using the proper version? If that is your assertion, do tell me how you've come by that information.

(You did notice that the author of the essay and the author of the video I linked to are not the same person, and that neither of them share a nym with me, yes?)

Re: ML promises to be profoundly weird

#238

Post by wat10000 »

arjie wrote: post #216
wat10000 wrote: post #207

Earlier quoted context omitted.

You do it as a hobby, that's fine. Some people do it for a living. And while they aren't owed a living doing that specific thing, it is going to be a big problem for them if they can't make money at it anymore. I'm sure plenty of people feel the same way about software. They make software as a hobby and don't care about remuneration or credit. Meanwhile I write software for my day job and losing the ability to make m…

Ah, I see. It’s just straightforward protectionism like dockworkers opposing automation and so on. That I do comprehend, in fact. I write software too and I may no longer be able to just do it in the old way. Pretty scary world but also exciting. I can’t imagine trying to restrict LLM software writers on that basis but I can comprehend it as simply self-interest. Fair enough.

Do you make money writing software? I bet you either try to restrict LLM usage or assign your rights to an employer who does. Putting code in the public domain is pretty rare, and extremely rare for paid work.

Re: ML promises to be profoundly weird

#239

Post by simianwords »

simoncion wrote: post #237
simianwords wrote: post #232

Earlier quoted context omitted.

This is classic case of not using the proper version. Use the thinking version gpt5.4 (text) and tell me if it bullshits. Surely you must be able to find at least one example no?

To be clear, is your assertion that apyhr was also not using the proper version? If that is your assertion, do tell me how you've come by that information. (You did notice that the author of the essay and the author of the video I linked to are not the same person, and that neither of them share a nym with me, yes?)

Hi, my position on the issue is that LLMs are powerful but may make mistakes in long context problems like coding (which the harness solves by feedback). But makes close to no (undergrad level) mistakes in questions that fit 2-3 pages. For you personally: do you believe me on this specific part on 2-3 pages?

I don't know what aphyr did and tbh his whole screed on LLMs make me feel he didn't use it properly or at least coming from a bad faith angle.

That's why I'm asking you (and others). Please come up with a text prompt spanning Surely the implication of such a screed is that it should be super simple to find at least one example of it clearly bullshitting in my constraint, no? Or am I interpreting the post in a bad faith way?

Re: ML promises to be profoundly weird

#240

Post by hackinthebochs »

lamasery wrote: post #190
erichocean wrote: post #125

> Models do not (broadly speaking) learn over time. They can be tuned by their operators, or periodically rebuilt with new inputs or feedback from users and experts. Models also do not remember things intrinsically: when a chatbot references something you said an hour ago, it is because the entire chat history is fed to the model at every turn. Longer-term “memory” is achieved by asking the chatbot to summarize a con…

I'm struggling to reckon how that can even possibly be true, unless we're counting automation of the "dumping that shorter summary into the input of every run" thing. I can imagine it being true with models so small that each user could afford to have their own, but not with big shared models like what're getting used for all the major services. Is that what you mean?

I see nothing to preclude a foundation model being augmented by a smaller model that serializes particulars about an individuals cumulative interaction with the model and then streamlines it into the execution thread of the foundation model.
Post reply on HN