261–270 of 533 posts
I've been stuck in loop all week where a coworker asked me to review some methodology documents that are clearly generated by Claude (which is fine in this case) but every time I sit down my brain can't get through 2 sentences without feeling like I'm reading nothing. It actually feels like the content, which isn't empty, has no meaning. So I go on to another assignment and come back later and the same thing happens.…
In high school, a teacher gave me a copy of "How To Read Better And Faster" which teaches you speed reading. This came in very handy in college. I find that when I try to speed read modern human writing, there are often errors (like missing or misused words) or awkward expressions that I do have to slow down and think harder a lot to really parse it. With AI writing, it's sort of self redundant and the information de…
That's interesting. You're saying speed reading helps you grasp information density of text? As someone who hasn't practiced speed reading, how does that happen? Is it something about the way your brain tries to connect ideas from different parts of the text? Or the redundancy making the signal more stable?
pre-read is just looking at how long it is in the headings, and planning out what chapters to focus on if it was a text book. (its sort of iterative, you do a pre-read for the whole book, and then for each section you break it into)
The fast read you try to read only with your eyes, sweeping your eyes across multiple words at the same time, suppressing the urge to say the words to yourself in your head.
iirc the how to read better and faster book even had a cardboard mask you put on the page to practice the sweeping, and some pages that were laid out weird to try to teach you how to do it.
Some ai text just seems really easy to speed read, like if it's tuned for an easy reading level. In PRs some ai seems like it's arguing over weird flex technical details and really starts torturing the language in a way that makes it the opposite of easy to read.
Earlier quoted context omitted.
It's just filled to the brim with relations between things. It's good at searching a very large meaning space and create correlations. What it does is to cover great distances and find related things in that large space which needs a long time and large corpus of knowledge to find the connection. This is not intelligence. It's just a good correlation engine with a very big albeit lossy database of things.
Intelligence is compression, compression requires subtraction, and for some reason LLMs are not good at subtracting. To create a coherent model you kinda have to subtract correlations until only the essential parts are still there. What I don't understand is why LLMs haven't been able to do this yet, if it's the harness or some orchestration layer above the LLM that is needed. Because fundamentally if you can identif…
The issue isn’t really harness vs. no harness. IMO it’s about the lack of an internally generated sense of what to attend to. Yes, the KV cache accumulates state and its “attention” (if you can even call it that) changes with context. We’ve even managed to /kinda/ close the loop with agentic tool calling and ‘memory’ systems, but these just close the loop at the level of behavior rather than disposition. All agentic harnesses do is make an LLM responsive to the consequences of its actions without changing the tendencies by which it determines what to retain or avoid.
The ghost you can’t escape from at this point is the origin of that relevance. Where does the pull toward one thing mattering over another actually come from? If you ran Fable 5 on a Turing machine and rewound the tape to the exact same state with the exact same input (incl. PRNG seed), it would spit out the same output every time.
Everyone’s trying to outrun this problem by training more often or increasing model sizes. But all this does is inform your model, from the outside(!), what constitutes a better state. The thing that’s actually doing the determining remains unchanged. Congratulations, you’ve scaled the transition function and tape of your Turing machine until it requires every watt generated by ERCOT, and it still cannot, for the life of it, tell you why it should give a shit.
A trained model generating output from weights, a seed, and some context effectively has next-state that’s a total function of those three things. Whatever behavior appears as ‘selecting what is relevant’ is, underneath, just a transition rule executing, no matter how sophisticated or creative the output looks. It can be fully accounted for by what was fixed before it started executing. Which means whatever criterion it uses for determining what matters was inherited from a structure that was already in place before it encountered the situation.
No amount of pruning or post-training can fix this. These approaches just replace one externally supplied criterion with another. For a system to be truly adaptable, there would have to be some criterion by which it treats one possible change as preferable to another, and that criterion itself would have to come from... somewhere. You can even change your conception of ‘improvement’ (e.g. parameter count, harnesses, self-modification, hell, even its ability to spit out shitty best-selling romance novels onto Amazon) and you still haven’t explained where the normative distinction comes from. Every layer of this problem has its root in a preference that was supplied from somewhere else.
I genuinely don’t know if this issue bottoms out anywhere, at least for the way we currently build these systems. Perhaps the solution is still computable, maybe? Who knows what that would even look like. But I’m fairly confident that it isn’t a bigger tape. I hope nobody solves this in the near future because, well, I’d like to have a job...
Earlier quoted context omitted.
This comment thread was started with discussions of AI doing a bad job at a task (communication). Doesn't the Chinese Room posit an AI good at the task of communication?
The Chinese Room mainly just posits a room that passes the Turing Test, which LLMs do pretty well outside of outright adversarial situations.
Earlier quoted context omitted.
The "something deeply wrong" part about AI, that even most technology enthusiasts evidently do not seem to grasp, is that it is still fundamentally a statistical model — an algorithmic construct — and does not possess any real intelligence or critical thought whatsoever. No matter how much investors and tech companies want you to believe that they are on the verge of super intelligence, nothing I've seen to date can…
> ... including the "novel" math solutions, all of which appear to just be "a composition of solutions humans have developed and documented elsewhere" upon deeper inspection. But that is precisely what human mathematicians do, prove new theorems by combining ones proven earlier. I don't see any fundamental difference in functionality between human intellectual contributions vs performant ML ones (LLM or otherwise). W…
I've been stuck in loop all week where a coworker asked me to review some methodology documents that are clearly generated by Claude (which is fine in this case) but every time I sit down my brain can't get through 2 sentences without feeling like I'm reading nothing. It actually feels like the content, which isn't empty, has no meaning. So I go on to another assignment and come back later and the same thing happens.…
Holy shit, someone putting it into words
Earlier quoted context omitted.
I just can't accept that it possesses no intelligence. It is not equivalent to human intelligence, obviously, but how can a system without some semblance of rational thinking solve open math problems? Even composing earlier human work into something novel requires intelligence and understanding on some level.
While being very capable, AI is missing something required for true intelligence and I struggle to explain exactly what it is I see missing. It's not really "creativity" because much of that always was derivative in my opinion. And LLMs are (for some definition of the word) fairly creative as far as taking known elements and re-arranging them. I think what is missing is sort of a world model building capability. As h…
Yes, models posses intelligence, but it is not a true one.
Then you claim that models do not posses world-building capabilities. But this is simply not true. Even ignoring the whole subgenre of scientific papers on exactly that subject, it is not that hard to build some hypothetical scenarios, big or small, and then witness the ease with which models do navigate those worlds.
That last image is bizarre. The quiche, cream and even the salad look like they've been given the trypophobia treatment. Which might even make sense, because there were always (still are?) those horrible ads in the chumbox area of news sites that used trypophobia and other creepy body-horror stuff to get you to click. [1] So maybe the hope is that you don't really look closely at the quiche, but some reptilian party…
It doesn't look like a quiche but more like a cake to me, and the top would be torched meringue, not mold. Although it's probably some weird ai mix of quiche and cake.
I've been stuck in loop all week where a coworker asked me to review some methodology documents that are clearly generated by Claude (which is fine in this case) but every time I sit down my brain can't get through 2 sentences without feeling like I'm reading nothing. It actually feels like the content, which isn't empty, has no meaning. So I go on to another assignment and come back later and the same thing happens.…
Worse, I noticed that people in an office environment themselves have adopted a more speculative, communication style.
In the past, people remembered what was said and would draw attention to discrepancies. I could trust what people said.
Nowadays it's like; someone can say one thing one day and the opposite the next day (through convoluted language) and nobody bats an eyelash. Or sometimes someone will agree with me but then what they say immediately after reveals that they didn't understand the essence of my point at all. I didn't notice these things 5 years ago.
I guess this is what AI researchers refer to as 'model collapse' - it seems to affect people too though...