41–48 of 48 posts
AI (assisted?) summary of the March 2025 Anthropic paper https://transformer-circuits.pub/2025/attribution-graphs/bio...
just curious.. are there languages that are better or more efficient to build LLM's with other than English?
just curious.. are there languages that are better or more efficient to build LLM's with other than English?
For some definitions of better, yes. Chinese is more token efficient for representing fixed text, for example, although this does not always lead to better performance on downstream tasks.
"lack of metacognitive insight" is interesting, because I have observed people acting this way too. I have even observed it in myself
I believe it's a great deal worse than that. All the metacognitive insight we do have may just be confabulation and we are fooled into believing that we have it because the process for conjuring it is good at finding a plausible answer. When you read about and observe the split-brain patient experiments the appropriate response is abject horror at the implications.
I don't recall being promised a black box. Are we certain llms didn't write this article and just came up with one of those It's Not Whatever-random-thing pithy zinger kind of things that they're prone to?
This was the mantra of applied machine learning c. 2010 - 2024 for anyone paying attention. No longer the case.
Earlier quoted context omitted.
They’re the language part of the puzzle, which seems to require some basic world modeling but it can’t make novel models unless there’s an example in its training data. I think engineering and mathematical thought requires spatial reasoning, when I model problems I see them as 3D shapes. Like the economy is a series of tubes that money flows through and collect in buckets, programming state is little boxes that hold…
Most LLMs are multimodal now, able to map visual concepts to language and vice versa. If OpenAI's recent Erdos solution was faking math, it faked it very well.
I can’t evaluate the Erdos solution personally, but both math and software have many problems that are some combination of other problems and since it can get instant verification feedback it can try millions of permutations to discover the right solution. This is valuable, I’m not dismissing it, but I think there’s another tier of harder problems that I don’t believe LLMs can solve and it will require some further theoretical breakthroughs to get there.
>On the Biology of a Large Language Model Biology? Anthropic really needs to stop anthropomorphizing these things so much. I'm with Dijkstra on this one. I know they do it as a sort of marketing but still...
I didn't see him mentioned in the article and I can't recall what he ever said about biology before..
>On the Biology of a Large Language Model Biology? Anthropic really needs to stop anthropomorphizing these things so much. I'm with Dijkstra on this one. I know they do it as a sort of marketing but still...
Why, what did Dijkstra have to say about biology vs computer systems? I didn't see him mentioned in the article and I can't recall what he ever said about biology before..