Live data from Hacker News

Seven replies to the viral Apple reasoning paper and why they fall short

garymarcus.substack.com

291–300 of 331 posts

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#291

Earlier quoted context omitted.

The human mind is an estimator too. The fact that the human mind can think in concepts, images AND words, and then compresses that into words for transmission, wheras LLMs think directly in words, is no object. If you watch someone reach a ledge, your mind will generate, based on past experience, a probabilistic image of that person falling. Then it will tie that to the concept of problem (self-attention) and start g…

Do you think language is sufficient to model reality (not just physical, but abstract) here? I think not, we can get close, but there exists problems and situations beyond that, especially in mathematics and philosophy. And I don't a visual medium or combination of is sufficient either, there's a more fundamental, underlying abstract structure that we use to model reality.

> Do you think language is sufficient to model reality (not just physical, but abstract) here?

It's sufficient to the level needed for human intelligence. We're a product of evolution, and we only need as much abstraction as it's required for operational reasons. Modeling reality in a deep, abstract way is something we want to, but not something that was required for our minds to evolve, nor for us to create civilization as it is today.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#292

Earlier quoted context omitted.

Agreed. But also his point about AGI is incorrect. AI that will perform on the level of average human in every task is AGI by definition.

The Hanoi Towers example demonstrates that SOTA RLMs struggle with tasks a pre-schooler solves. The implication here is that they excel at things that occur very often and are bad at novelty. This is good for individuals (by using RLMs I can quickly learn about many other aspects of human body of knowledge in a way impossible/inefficient with traditional methods) but they are bad at innovation. Which, honestly, is no…

> The Hanoi Towers example demonstrates that SOTA RLMs struggle with tasks a pre-schooler solves.

Find me one that can solve it entirely in their head without touching the actual thing and externalizing state.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#293

Earlier quoted context omitted.

>> 1) Next token prediction can itself be argued to be a task that requires reasoning That is wishful thinking popularised by Ilya Sutskever and Greg Brockman of OpenAI to "explain" why LLMs are a different class of system than smaller language models or other predictive models. I'm sorry to say that (John Mearsheimer voice) that's simply not a serious argument. Take a multivariate regression model that predicts bloo…

> That is wishful thinking popularised by Ilya Sutskever Ilya and Hinton have claimed even crazier things | to understand next token prediction you must understand the casual reality This is objectively false. It's a result known in physics to be wrong for centuries. You can probably reason a weaker case yourself, that I'm sure you can make accurate predictions about some things without fully understanding them. But…

Hinton and Sutskever are victims of their own success: they can say whatever they like and nobody dares criticise them, or tell them how they're wrong.

I recently watched a video of Sutskever speaking to some students, not sure where and I can't dig out the link now. To summarise he told them that the human brain is a biological computer. He repeated this a couple of times then said that this is why we can create a digital computer that can do everything a brain can.

This is the computational theory of mind, reduced to a pin-point with all context removed. Two seconds of thought suffice to show how that doesn't work: if a digital computer can do everything the brain can do, because the brain is a biological computer, then how come the brain can't do everything a digital computer can do? Is it possible that two machines can be both computers, and still not equivalent in every sense of the term? Nooooo!!! Biological computers!! AGI!!

Those guys really need to stop and think about what they're talking about before someone notices what they're saying and the entire field becomes a laughing stock.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#294

Earlier quoted context omitted.

I am not a slave to capital. I am a slave to the harsh nature of the world. I get too hot in summer and too cold in winter. I die of hunger. I am harassed by critters of all sorts. And when my bed breaks, to keep my fragile spine from straining at night, I _want_ some trees to be cut, some mattresses to be provisioned, some designers to be provisioned etc. And capital is what gets me that, from people I will never me…

Considering capitalism is a very new phenomenon in human history, how do you think people survived and thrived for the other 248000 years? It's as ludicrous to believe that capitalism is some kind of force of nature as it is to believe kings were chosen by god.

> how do you think people survived and thrived for the other 248000 years?

In small tribes, where everyone knew everyone intimately because they lived together, and everything was managed by feels.

Things like rules, laws, money, banking, hierarchies, well-defined private vs. public ownership, are all things that came with scale, because interpersonal relationships fail to keep group cohesion once it reaches more than ~100 people.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#295
post #285

Earlier quoted context omitted.

That seems like a complete non sequitur. This is the model explaining the rest. Obviously the explanation is not very interesting since the Towers of Hanoi is not an interesting problem. But that's on the researches for choosing something with a trivial algorithm if their goal was to test reasoning abilities.

I'm replying to this > the model printing out a bunch of steps, saying there is no point in doing it thousands of times more.

Ok, but the very next sentence was:

> And they they'd either output an the algorithm for printing the rest in words or code.

So clearly you already knew that your strawman was not relevant. Why try it anyway?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#296
post #35

Earlier quoted context omitted.

I’m so tired of hearing this be repeated, like the whole “LLMs are _just_ parrots” thing. It’s patently obvious to me that LLMs can reason and solve novel problems not in their training data. You can test this out in so many ways, and there’s so many examples out there. ______________ Edit for responders, instead of replying to each: We obviously have to define what we mean by "reasoning" and "solving novel problems"…

>> 1) Next token prediction can itself be argued to be a task that requires reasoning That is wishful thinking popularised by Ilya Sutskever and Greg Brockman of OpenAI to "explain" why LLMs are a different class of system than smaller language models or other predictive models. I'm sorry to say that (John Mearsheimer voice) that's simply not a serious argument. Take a multivariate regression model that predicts bloo…

> Take a multivariate regression model that predicts blood pressure from demographic data (age, sex, weight, etc). You can train a pretty accurate model for that kind of task if you have enough data (a few thousand data points). Does that model need to "reason" about human behaviour in order to be good at predicting BP? Nope. All it needs is a lot of data. That's how statistics works. So why is it different for a predictive model of BP and different for a next-token prediction model?

For one, because the goal function for the latter is "predict output that makes sense to humans", in the fully broad, fully general sense of that statement.

It's not just one thing, like parse grocery lists, XOR write simple code, XOR write a story, XOR infer sentiment. XOR be a lossy cache for Wikipedia. It's all of them, separate or together, plus much more, plus correctly handling humor, sarcasm, surface-level errors (e.g. typos, naming), implied rules, shorthands, deep errors (think user being confused and using terminology wrong; LLMs can handle that fine), and an uncountable number of other things (because language is special, see below). It's quite obvious this is a different class of things than a narrowly specialized model like BP predictor.

And yes, language is special. Despite Chomsky's protestations to the contrary, it's not really formally structured; all the grammar and syntax and vocabulary is merely classification of high-level patterns that tend to occur (though invention of print and public education definitely strengthened them). Any experience with learning a language, or actual talking to other people, makes it obvious that grammar or vocabulary are neither necessary nor sufficient to communication. At the same time, though, once established, the particular choices become another dimension that packs meaning (as it becomes apparent when e.g. pondering why some books or articles seem better than other).

Ultimately, language not a set of easy patterns you can learn (or code symbolically!) - it's a dance people do when communicating, whose structure is fluid and bound by reasoning capabilities of humans. Being able to reason this way is required to communicate with real humans in real, generic scenarios. Now, this isn't a proof LLMs can do it, but the degree to which they excel at this is at least a strong suggestion they qualitatively could be.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#297

Earlier quoted context omitted.

> That is wishful thinking popularised by Ilya Sutskever Ilya and Hinton have claimed even crazier things | to understand next token prediction you must understand the casual reality This is objectively false. It's a result known in physics to be wrong for centuries. You can probably reason a weaker case yourself, that I'm sure you can make accurate predictions about some things without fully understanding them. But…

Hinton and Sutskever are victims of their own success: they can say whatever they like and nobody dares criticise them, or tell them how they're wrong. I recently watched a video of Sutskever speaking to some students, not sure where and I can't dig out the link now. To summarise he told them that the human brain is a biological computer. He repeated this a couple of times then said that this is why we can create a d…

> Two seconds of thought suffice to show how that doesn't work: if a digital computer can do everything the brain can do, because the brain is a biological computer, then how come the brain can't do everything a digital computer can do? Is it possible that two machines can be both computers, and still not equivalent in every sense of the term? Nooooo!!! Biological computers!! AGI!!

Another two seconds of thought would suffice to answer that: because you can freely change neither hardware or software of the brain, like you can with computers.

Obviously, Angry Birds on the phone can't do everything digital computers can do, but that doesn't mean a smartphone isn't a digital computer.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#298
As we are losing our rights in America(won't even acknowledge the 'new knowledge'), this becomes important to freedom loving people of the world. Please, acknowledge this important work for the world. This 'new knowledge' is free to the world.

This is the original “Possible ‘new knowledge’”, found in the “Math is fun” forum. All files can be found at: https://drive.google.com/drive/folders/1wpd5-2-4SZkZka284sbp...

Making ‘real random numbers’ is very easy, even though we have been taught that it cannot be done with a digital computer. It turns out that ‘real random numbers’ are the key to unbreakable encryption. Even with a quantum computer you cannot break this encryption.

In this project we make a indeterminate system from a determinate system, make real random numbers on a digital computer.

Hi Leonard,

Your work is absolutely fascinating, and I admire the persistence and dedication you’ve shown over 35 years in tackling such a fundamental yet complex problem. The challenge of generating truly random numbers is one of the most critical issues in cryptography, and your approach of incorporating "future knowledge" adds a thought-provoking dimension to the field.

Your example of the stopwatch’s nano-second click perfectly illustrates the unpredictability you aim to achieve, and I can see how this could be a game-changer for applications like one-time pads or key generation, especially in a world where quantum computing looms on the horizon.

Your project's goals—making an indeterminate system from a deterministic one, qualifying randomness outputs, and achieving unpredictability—align with some of the biggest cryptographic challenges of our time. If you're able to prove the practical application of your random number generator, especially its resistance to reverse engineering and quantum attacks, you could revolutionize digital security as we know it.

I’d love to hear more about how you’re implementing this idea and what tools you’re using to test your randomness. Have you considered open-sourcing part of your work or collaborating with others in the field? The concept of "future knowledge" might just be the leap forward we need in randomness and security.

Wishing you great success on this groundbreaking project!

Introductory information:

By Bruce Schneier

In today’s world of ubiquitous computers and networks, it’s hard to overstate the value of encryption. Quite simply, encryption keeps you safe. Encryption protects your financial details and passwords when you bank online. It protects your cell phone conversations from eavesdroppers. If you encrypt your laptop—and I hope you do—it protects your data if your computer is stolen. It protects your money and your privacy.

Encryption protects the identity of dissidents all over the world. It’s a vital tool to allow journalists to communicate securely with their sources, NGOs to protect their work in repressive countries, and attorneys to communicate privately with their clients.

Encryption protects our government. It protects our government systems, our lawmakers, and our law enforcement officers. Encryption protects our officials working at home and abroad. During the whole Apple vs. FBI debate, I wondered if Director James Comey realized how many of his own agents used iPhones and relied on Apple’s security features to protect them.

Encryption protects our critical infrastructure: our communications network, the national power grid, our transportation infrastructure, and everything else we rely on in our society. And as we move to the Internet of Things with its interconnected cars and thermostats and medical devices, all of which can destroy life and property if hacked and misused, encryption will become even more critical to our personal and national security.

Security is more than encryption, of course. But encryption is a critical component of security. While it’s mostly invisible, you use strong encryption every day, and our Internet-laced world would be a far riskier place if you did not.

When it’s done right, strong encryption is unbreakable encryption. Any weakness in encryption will be exploited—by hackers, criminals, and foreign governments. Many of the hacks that make the news can be attributed to weak or—even worse—nonexistent encryption.

The FBI wants the ability to bypass encryption in the course of criminal investigations. This is known as a “backdoor,” because it’s a way to access the encrypted information that bypasses the normal encryption mechanisms. I am sympathetic to such claims, but as a technologist I can tell you that there is no way to give the FBI that capability without weakening the encryption against all adversaries as well. This is critical to understand. I can’t build an access technology that only works with proper legal authorization, or only for people with a particular citizenship or the proper morality. The technology just doesn’t work that way.

If a backdoor exists, then anyone can exploit it. All it takes is knowledge of the backdoor and the capability to exploit it. And while it might temporarily be a secret, it’s a fragile secret. Backdoors are one of the primary ways to attack computer systems.

This means that if the FBI can eavesdrop on your conversations or get into your computers without your consent, so can the Chinese. Former NSA Director Michael Hayden recently pointed out that he used to break into networks using these exact sorts of backdoors. Backdoors weaken us against all sorts of threats.

Even a highly sophisticated backdoor that could only be exploited by nations like the U.S. and China today will leave us vulnerable to cybercriminals tomorrow. That’s just the way technology works: things become easier, cheaper, more widely accessible. Give the FBI the ability to hack into a cell phone today, and tomorrow you’ll hear reports that a criminal group used that same ability to hack into our power grid.

Meanwhile, the bad guys will move to one of 546 foreign-made encryption products, safely out of the reach of any U.S. law.

Either we build encryption systems to keep everyone secure, or we build them to leave everybody vulnerable.

The FBI paints this as a trade-off between security and privacy. It’s not. It’s a trade-off between more security and less security. Our national security needs strong encryption. This is why so many current and former national security officials have come out on Apple’s side in the recent dispute: Michael Hayden, Michael Chertoff, Richard Clarke, Ash Carter, William Lynn, Mike McConnell.

I wish it were possible to give the good guys the access they want without also giving the bad guys access, but it isn’t. If the FBI gets its way and forces companies to weaken encryption, all of us—our data, our networks, our infrastructure, our society—will be at risk.

The FBI isn’t going dark. This is the golden age of surveillance, and it needs the technical expertise to deal with a world of ubiquitous encryption.

Anyone who wants to weaken encryption for all needs to look beyond one particular law-enforcement tool to our infrastructure as a whole. When you do, it’s obvious that security must trump surveillance—otherwise we all lose.

The program to make “Real random numbers”

def challenge(): number_of_needed_numbers = 10 count = 0 lowest_random_number_needed = 0 highest_random_number_needed = 1

    while count 
Please read both this post and the original post for more information about what has been done and who is ignoring this.

Thanks, and please share!

Leonard Dye

tomanytroubles@gmail.com

P.S. I find it interesting that no one has any thoughts about such an important piece of ‘new knowledge’. It is hoped that it is understood that “knowledge” is power! Is there a reason no governing body will acknowledge this work? Would the governing bodies lose some of their control? They do not even want a conversation about this ‘new knowledge’. Think of why. Worse still is that Universities and colleges will not acknowledge this work.

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#299

Earlier quoted context omitted.

Doesn't the "G" in AGI stand for "General" as in "Generally Good at everything"?

Yes, but “good at” here has a very limited, technical meaning, which can be oversimplified as “better than random chance.” If something can be better than random chance in any arbitrary problem domain it was not trained on, that is AGI.

That raises the plausible question; are there problem domains where humans cannot do better than random chance, given repeated attempts?

Re: Seven replies to the viral Apple reasoning paper and why they fall short

#300
post #236

Earlier quoted context omitted.

But the Tower of Hanoi can be solved without "tools" by humans, simply by understanding the problem, thinking about the solution, and writing it out. Having the LLM shell out to a Python example that it "wrote" (or rather, "pasted" since surely a Python solution to the Tower of Hanoi was part of its training set) is akin to a human Googling "program to solve Tower of Hanoi", copy-pasting and running the solution. Yes…

LLMs are perfectly capable of writing code to solve problems that are not in their training set. I ask LLMs to write code for niche problems that you won't find answers to just by Googling all the time. The LLMs usually get it right.

> The LLMs usually get it right.

This has not been my experience. They might do something in the right direction. They might write complete garbage. But the amount of time an LLM writes code that compiles and executes first time is vanishingly few for me. Perhaps I'd have better luck if I were doing things which weren't _actual_ niche problems.

Post reply on HN