Viewing profile — Vetch

What constitutes serious work and how seriously have you tried to do serious work with them? While those trying to claim a 30B dense model can match Opus 4.6 are engaging in either…

Interesting thought to consider in principle but fails because gorilla brains continued to evolve too, just along a different path. They're not snapshots of ancestral species locke…

This sounds very like Licklider's essay on Intelligence Amplification: Man Computer Symbiosis, from 1960: > Men will set the goals and supply the motivations, of course, at least i…

The relu/if-then-else is in fact centrally important as it enables computations with complex control flow (or more exactly, conditional signal flow or gating) schemes (particularly…

This is essentially what any relu based neural network approximately looks like (smoother variants have replaced the original ramp function). AI, even LLMs, essentially reduce to a…

I'm not sure that's the fully right mental model to use. They're not searching randomly with unbounded compute nor selecting from arbitrary strategies in this example. They are bot…

Then the point still stands, this makes things even worse given that it's adding its own hallucinations on top, instead of simply relaying the content or idealistically, identifyin…

Being tall doesn't automatically make you good or dominant at basketball, you can even be too tall. Wemby might just be at that threshold, but the unusual thing about him is his de…

It's an artifact of post-training approach. Models like kimi k2 and gpt-oss do not utter such phrases and are quite happy to start sentences with "No" or something to the tune of "…

You are right and the idea of LLMs as lossy compression has lots of problems in general (LLMs are a statistical model, a function approximating the data generating process). Compre…

Unless you're also writing your own graphics and game engine from scratch, if you're making a truly novel and balanced game, then it should not be possible to crank out code with A…

Why would the proportion of high quality games increase? The number yes, but I expect not the proportion. Lowering the entry barrier means more people who have spent less time honi…

But why isn't this merely papering over a more fundamental issue with how these models are "aligned"? LLMs are, for example, not inherently sycophantic. kimi k2 and o3 are not, and…

The brain is certainly vastly more energy efficient at inference than LLMs on GPUs. But it looks like you're trying to make a different argument, that an LLM can spend less energy …

That math is for random projections? Note that JL lemma is a worst case guarantee and in practice, there's a lot more distortion tolerance than the given bounds would suggest. Conc…

Compounding with learn and iterate, humans also build abstractions which significantly shorten the number of steps required. These are more expressive programming languages, compil…

> he's inability to see its application to modern compute held the field back by years. I find Schmidhuber's claim on GANs to be tenuous at best, but his claim to have anticipated …

The non-o-series models from OpenAI and non-Opus (although I have not tried the latest, so it's possible that it too joins them) from Anthropic are cloyingly sycophantic, with ever…

This contains a common misstep (or misgeneralization of an analogy) among those who are much more familiar with computers than with the brain. The brain is not digital and concepts…

I think the point remains that few have been able to catch up to OpenAI. For a while it was just Anthropic. Then Google after failing a bunch of times. So, if we relax this to LLMs…

There is skill to it but that's certainly not the only relevant variable involved. Other important factors are: Language: Syntax errors rise, and a common form is the syntax of a m…

LeBron is one of the rare individuals at that intersection of high athleticism and mental capability. It's why at the age of 40, well past his athletic prime, he's still a top NBA …

The essence of the article is that self-correction exists as a nascent ability in base models already (more robustly in some like Qwen than others). This is highly reminiscent of C…

I'm also ashamed to say I've also never seen any of his movies and TV series but this still hits hard because of his influence on some my most cherished fictional properties. These…

This is an interesting proposition. Have you tested this with the best open LLMs?