301–310 of 313 posts
I can reason. Sometimes. It's very hard. My buddy Deepseek can't. This is like the scene in Blue's Clues where the answer is obvious and the kids are yelling but blue can't see it. Facts abound, but not conclusions based on those facts
There's a reachable intermediate step on the way before reasoning, and that's "keeping the plot". Not losing the line of thought.
> For example, transformer-based language models can't reliably do multi-digit arithmetic because they have no number sense, only statistical patterns over digit tokens. I asked Opus 4.6 to multiply 50,651 x 895,707. It immediately responded: 45,368,055,057. This was wrong, but very close (good to about 10^-5 relative error). I then asked it how it arrived at that answer, and whether it had used any tool calls. It sa…
This is in every way an argument for the author’s point.
Modern AI came about from mimicking how natural neurons worked, and we can't get to AGI without also mimicking higher-level brain structures such as the neocortex neural column.
Earlier quoted context omitted.
> Lets all arbitrarily agree AGI is here. I can't even be bothered discussing what the definition of AGI is. There is a definition of AGI the AI companies are using to justify their valuation. It's not what most people would call AGI but it does that job well enough, and you will care when it arrives. They define it as an AI that can develop other AI's faster than the best team of human engineers. Once they build one…
May I ask, what experiences are you personally having with LLMs right now that is leading you to the conclusion that they will become "intelligent" enough to identify, organise, and build advancing improvements to themselves, without any human interaction in the near future (1 - 2 years lets say)?
None, as I don't develop LLM's.
I wasn't saying I think they will succeed, but I think it is worth noting their AGI ambitions are not as grand as the term implies. Nonetheless, if they achieve them, the world will change.
Earlier quoted context omitted.
> expert opinions can be the slowest to update on the news that their specific domain may have, in hindsight, have been the wrong horse. It's the quote about it being difficult to believe something that your income requires to be false, but instead of income it can be your whole legacy or self concept Not sure I follow. Are you saying that AI researchers would be out of a job if scaling up transformers leads to AGI?…
People have entire careers promoting incorrect ideas. Oxycontin, phrenology, the windows operating system. Reconciling your self-concept with the negative (or fruitless) impacts of your life's work is difficult. It can be easier to deny or minimize those impacts.
Or is your reasoning that they will be upset about not having invented it themselves (similar to those conspiracy theories about, the cure for cancer existing but scientists withholding it so they can keep doing treatment research)?
Earlier quoted context omitted.
May I ask, what experiences are you personally having with LLMs right now that is leading you to the conclusion that they will become "intelligent" enough to identify, organise, and build advancing improvements to themselves, without any human interaction in the near future (1 - 2 years lets say)?
> May I ask, what experiences are you personally having with LLMs right now that is leading you to the conclusion that they will become "intelligent" enough to identify, organise, and build advancing improvements to themselves, without any human interaction in the near future (1 - 2 years lets say)? None, as I don't develop LLM's. I wasn't saying I think they will succeed, but I think it is worth noting their AGI amb…
Earlier quoted context omitted.
Thanks for reading, and I really appreciate your comments! > who feed their produced tokens back as inputs, and whose tuning effectively rewards it for doing this skillfully Ah, this is a great point, and not something that I considered. I agree that the token feedback does change the complexity, and it seems that there's even a paper by the same authors about this very thing! https://arxiv.org/abs/2310.07923 I'll ha…
You seem to have a lot of theoretical knowledge on this, but have you tried Claude or codex in the past month or two? Hands on experience is better than reading articles. I've been coding for 40 years and after a few months getting familiar with these tools, this feels really big. Like how the internet felt in 1994.
Fun observation - almost every coding harness (claude code, cursor, codex) uses a find/replace tool as the primary way of interacting with code. This requires the agent to fully type out the code it's trying to edit, including several lines of context around the edit. This is really inefficient, token wise! Why does it work this way? Because the LLMs are really bad at counting lines, or using other ways of describing a unique location in the file.
I've experimented with providing a more robust dsl for text manipulation https://github.com/dlants/magenta.nvim/blob/main/node/tools/... , and I do think it's an improvement over just straight search/replace, but the agents do tend to struggle a lot - editing the wrong line, messing up the selection state, etc... which is probably why the major players haven't adopted something like this yet.
So I feel pretty confident in my assessment of where these models are at!
And also, I fully believe it's big. It's a huge deal! My work is unrecognizable from what it was even 2 years ago. But that's an impact / productivity argument, not an argument about intelligence. Modern programming languages, IDEs, spreadsheets, etc... also made a fundamental shift in what being a software engineer was like, but they were not generally intelligent.
Earlier quoted context omitted.
> May I ask, what experiences are you personally having with LLMs right now that is leading you to the conclusion that they will become "intelligent" enough to identify, organise, and build advancing improvements to themselves, without any human interaction in the near future (1 - 2 years lets say)? None, as I don't develop LLM's. I wasn't saying I think they will succeed, but I think it is worth noting their AGI amb…
I mis-read. Thanks for clarifying :-)
> and you will care if/when it arrives.
Earlier quoted context omitted.
You seem to have a lot of theoretical knowledge on this, but have you tried Claude or codex in the past month or two? Hands on experience is better than reading articles. I've been coding for 40 years and after a few months getting familiar with these tools, this feels really big. Like how the internet felt in 1994.
I've been developing an ai coding harness https://github.com/dlants/magenta.nvim for over a year now, and I use it (and cursor and claude code) daily at work. Fun observation - almost every coding harness (claude code, cursor, codex) uses a find/replace tool as the primary way of interacting with code. This requires the agent to fully type out the code it's trying to edit, including several lines of context around th…
Incidentally, I saw an interesting article about exactly this subject a little ways back, using line numbers + hashes instead of typing out the full search/replace, writing patches, or doing a DSL, and it seemed to have really good success: