81–90 of 224 posts
It's simple. Don't ingest more than 40KB at a time into its LLM's RAG pipe and its hallucination goes way, way down. Preferably like not at the start and best not to do more than 40KB at a time at all. That's how I learned how to deal with nftables' 120KB parser_bison.y file by breaking them up into clean sections. All of a sudden, a fully-deterministic LL(1) full semantic pathway of nftables' CLI syntax appears befo…
Earlier quoted context omitted.
I have one hammer and I expect it to work on every nail and screw. If it's not a general hammer, what good is it now?
You don't need a "general hammer" - they are old fashioned - you need a "general-purpose tool-building factory factory factory": https://www.danstroot.com/posts/2018-10-03-hammer-factories
Earlier quoted context omitted.
You are making a wrong leap from non-deterministic process to uncontrollable result. Most of the parallel algorithms are non-deterministic. There might be no guarantee about the order of calculation or even sometimes the final absolute result. However, even when producing different final results, the algorithm can still guarantee characteristics about the result. The hard problem then is not to eliminate non-determin…
Life and a lot in our universe is non-deterministic. Some people assume science and mathematics are some universal truths rather than imperfect agreed upon understandings. Similarly many assume humans can be controlled through laws, penalties, prisons, propaganda, coercion, etc. But terrible things happen. Yes, if you set up the gutter-rails in your bowling lane, you can control the bowling ball unless it is thrown o…
Earlier quoted context omitted.
> So, that makes me come back to this question of what definition of reasoning do people use that reasoning models do not meet? The models can learn reasoning rules, but they are not able to apply them consistently or recognize the rules they have learned are inconsistent. (See also my other comment which references comments I made earlier.) And I think they can't without a tradeoff, as I commented https://news.ycomb…
Yes, so I think in this case we use different definitions of reasoning. You include reliability as a part of reasoning, whereas I do not. I would argue that humans are not 100% reliable in their reasoning, and yet we still claim that they can reason. So, even though I would agree that the reasoning of LLMs is much less reliable, careful, and thoughtful than smart humans, that does not mean that they are not reasoning…
You can argue that damaged toaster is still a toaster, conceptually. But if it doesn't work, then it's useless. As it stands, models lack ability to reason because they can fail to reason and you can't do anything about it. In case of humans, it's valid to say they can reason, because humans can at least fix themselves, models can't.
I'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
Earlier quoted context omitted.
That's the real deal. They say LLM are PhD-level. Despite billion dollars, PhD-LLMs sure are not contributing a lot solving known problems. Except of course few limited marketing stunts.
IMHO that's the key differentiator. You can give a human PhD an _unsolved problem_ in field adjacent to their expertise and expect some reasonable resolution. LLM PhD:s solve only known problems. That said humans can also be really bad problem solvers. If you don't care about solving the problem and only want to create paperwork for bureaucracy I guess you don't care either way ("My team's on it!") but companies that…
Terry Tao would disagree: https://mathstodon.xyz/@tao/114508029896631083
https://deepmind.google/discover/blog/alphaevolve-a-gemini-p...
The key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from som…
Earlier quoted context omitted.
Any example of that? One would think that predicting what comes next from an image is basically video generation, which works not perfect, but works somehow (Veo/Sora/Grok)
Here's one I made in Veo3.1 since gemini is the only premium AI I have access to. Using this image - https://www.whimsicalwidgets.com/wp-content/uploads/2023/07/... and the prompt: "Generate a video demonstrating what will happen when a ball rolls down the top left ramp in this scene." You'll see it struggles - https://streamable.com/5doxh2 , which is often the case with video gen. You have to describe carefully and…
Is that really the best title the authors could come up with? Up next: "Lawn mowers are good at cutting grass until they aren't"
I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confi…
I saw a meme that I think about fairly often: Great apes have learnt sign language, and communicated with humans, since the 1960's. In all that time they've never asked human questions. They've never tried to learn anything new! The theory is that they don't know that there are entities that know things they don't. I like to think that AI are the great apes of the digital world.
They don't have the dexterity to really sign properly