161–170 of 1001 posts
This is quite incredible Could you imagine if MS had convinced the govt back in the day, to require a special license to build an operating system (this blocking Linux and everything open)? It’s essentially what’s happening now, Except it is OpenAI instead of MS, and it is AI instead of Linux AI is the new Linux, they know it, and are trying desperately to stop it from happening
In other words, MS with extra steps.
Earlier quoted context omitted.
What do you think about the papers showing mathematical proofs that GNNs (i.e. GATs/transformers) are dynamic programmers and therefore perform algorithmic reasoning? The fact that these systems can extrapolate well beyond their training data by learning algorithms is quite different than what has come before, and anyone stating that they "simply" predict next token is severely shortsighted. Things don't have to be '…
Give me a break. Very interesting theoretical work and all, but show me where it's actually being used to do anything of value, beyond publication fodder. You could also say MLPs are proved to be universal approximators, and can therefore model any function, including the one that maps sensory inputs to cognition. But the disconnect between this theory and reality is so great that it's a moot point. No one uses MLPs…
There were two very noteworthy (Perhaps Nobel prize level?) breakthroughs in two completely different fields of mathematics (knot theory and representation theory) by using these systems.
I would certainly not call that "useless", even if they're not quite Nobel-prize-worthy.
Also, "No one uses GATs in systems people discuss right now" ... Transformerare GATs (with PE) ... So, you're incredibly wrong.
I see a lot of negative reactions, but I don't know the specific details of what is being proposed.
Earlier quoted context omitted.
What do you think about the papers showing mathematical proofs that GNNs (i.e. GATs/transformers) are dynamic programmers and therefore perform algorithmic reasoning? The fact that these systems can extrapolate well beyond their training data by learning algorithms is quite different than what has come before, and anyone stating that they "simply" predict next token is severely shortsighted. Things don't have to be '…
I just don't get how the average HN commenter thinks (and gets upvoted) that they know better than e.g. Ilya Sutskever who actually, you know, built the system. I keep reading this "it just predicts words, duh" rhetoric on HN which is not at all believed by people like Ilya or Hinton. Could it be that HN commenters know better than these people?
"So, you've thought about eternity for an afternoon, and think you've come to some interesting conclusions?"
Earlier quoted context omitted.
To be fair LLMs are predicting the next token. It's just that to get better and better predictions it needs to understand some level of reasoning and math. However it feels to me that a lot of this reasoning is brute forced from the training data. Like chatgpt gets some things wrong when adding two very large numbers. If it really knew the algorithm for adding two numbers it shouldn't be making them in the first plac…
And LLMs will never be able to reason about mathematical objects and proofs. You cannot learn the truth of a statement by reading more tokens. A system that can will probably adopt a different acronym (and gosh that will be an exciting development... I look forward to the day when we can dispatch trivial proofs to be formalized by a machine learning algorithm so that we can focus on the interesting parts while still…
Earlier quoted context omitted.
> Why is it so hard to hear this perspective? Like, genuinely curious. Because people have different definition of what intelligence is. Recreating the human brain in a computer would definitely be neat and interesting but you don't need that nor AGI to be revolutionary. LLMs, as perfect Chinese Rooms, lack a mind or human intelligence but demonstrate increasingly sophisticated behavior. If they can perform tasks bet…
I have a chain saw that can cut better than me, a car that can go faster, a computer that can do math better, etc. We've been doing this forever with everything. Building tools is what makes us unique. Why is building what amounts to a calculator/spreadsheet/CAD program for language somehow a Rubicon that cannot be crossed? Did people freak out this much about computers replacing humans when they were shown to be goo…
Imagine thinking that regression based function approximators are capable of anything other than fitting the data you give it. Then imagine willfully hyping up and scaring people who don't understand, and because it can predict words you take advantage of the human tendency to anthropomorphize, so it follows that it is something capable of generalized and adaptable intelligence. Shame on all of the people involved in…
I'm squarely in the "stochastic parrot" camp (I know it's not a simple markov model, but still, ChatGPT doesn't think), and it's clearly possible to interpret this as a grifting, but your argumentation is too simple. You're leaving out the essentials. These models do more than fitting the data given. They can output it in a variety of ways, and through their approximation, can synthesize data as well. They can output…
Imagine thinking that regression based function approximators are capable of anything other than fitting the data you give it. Then imagine willfully hyping up and scaring people who don't understand, and because it can predict words you take advantage of the human tendency to anthropomorphize, so it follows that it is something capable of generalized and adaptable intelligence. Shame on all of the people involved in…
Why is it so hard to hear this perspective? Like, genuinely curious. This is the first I hear of someone cogently putting this thought out there, but it seems rather painfully obvious -- even if perhaps incorrect, but certainly a perspective that is very easy to comprehend and one that merits a lot of discussion. Why is it almost nonexistent? I remember even in the hay day of crypto fever you'd still have A LOT of fo…
Because it's wrong and smart people know that.