881–890 of 1001 posts
Earlier quoted context omitted.
intelligence isn't a function unless you're talking about over every possible state of the universe.
Intelligence must inherently be a function unless there is a third form of cause-effect transition that can't be modelled as a function of determinism and randomness.
Earlier quoted context omitted.
GPT-4 still makes plenty of mistakes when programming that reveal that it doesn’t fully understand what it’s doing. It’s very good, but it doesn’t reach the level of human intellect. Yet. It is A and gets the G but fails somewhat on the I of AGI.
Humans also make mistakes, all the time.
This is easier to see with AI art. The artwork is very impressive but if the hand has the wrong number of fingers or the lettering is hilariously wrong, there’s a tendency to dismiss it.
Nobody complains that dall-e can’t produce artwork on par with Da Vinci because that’s not something we expect humans to do either.
For us to start considering these AIs “intelligent” they first need to nail what we consider “the basics”, no matter how hard those basics are for a machine.
Earlier quoted context omitted.
These guys smell so much like Tesla it's not even funny. Very impressive core tech, and genuinely advancing knowledge. But the hype train is just so powerful that the insane (to put it mildly) claims are picked up without any sense of critical thinking by seemingly intelligent people. They're both essentially cults at this point
Agreed, but IMHO it is sort of justified for Tesla. The size of its hype matches the size of the ICEs entrenchment in its moat. These have outsize influence on our economy, but climate change (and oil depletion) is quite inevitable. It takes irrational market cap to unseat that part of the economy being prisoner of its rent. And some allocators of capital have understood that.
Earlier quoted context omitted.
As someone with a CS background myself, I don't think this is what GP was talking about. Let's forget for a moment that stuff has to run on an actual machine. If you had to represent a quadratic equation, would you rather write: (a) x^2 + 5x + 4 = 0 (b) the square of the variable plus five times the variable plus four equals zero When you are trying to solve problems with a level of sophistication beyond the toy stuf…
There are some domains that are in the twilight zone between language and deductive, formal reasoning. I've been into genealogy last year. It's very often deductive "detective work": say there are four women in a census with the same name and place that are listed on a birth certificate you're investigating. Which of them is it? You may rule one out on hard evidence (census suggests she would have been 70 when the bi…
sounds like belief change systems (a bit) to me!
Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…
Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).
Earlier quoted context omitted.
The statement you quoted also holds for humans. Of those who can solve grade school math problems, very, very few can solve mathematical problems at a research level.
We're moving the goalposts all the time. First we had the Turing test, now AI solving math problems "isn't impressive". Any small mistake is a proof it cannot reason at all. Meanwhile 25% humans think the Sun revolves around the Earth and 50% of students get the bat and ball problem wrong.
I was talking to my (12 year old) son about parts of math he finds boring. He said that he thinks absolute value is absurdly easy and extremely boring. I asked him if there was anything that might make it more interesting, he said "maybe complex numbers". So I asked him "what would the absolute value of i+1 be?" he thinks for a little bit and says "square root of 2" and I ask him "what about the absolute value of 2i…
Sorry, can you explain this? To me, it makes sense to define abs(x) = sqrt(x^2) i.e. ignoring the negative solution enforces the positive result. Using that definition, abs(i+1) = sqrt((i+1)^2) = sqrt(i^2 + 2i + 1) = sqrt(-1 + 2i + 1) = sqrt(2i) != sqrt(2). The second example seems off in the same way (i.e. the answer should be sqrt(8i) instead of sqrt(8)). Am I missing something? Also, abs(i+2) = sqrt((i+2)^2) = sqr…
One trick about that inner product is that it need not be perfectly symmetric. To make it work on complex numbers we realize that we have to define it like p(a,b) = a . conj(b) where the . is normal multiplication and the conjugate operation reflects a complex number over the real line.
Now sqrt(p(i+1, i+1)) is sqrt((i+1) . (-i+1)) = sqrt(-i^2 + i - i + 1) = sqrt(2).
I’m skipping over a lot but I wanted to gesture toward where your intuition matches some well known concepts so that you could dive in more deeply. Also wanted to mention the conjugation trick to make your example work!
Seems like they have made progress in combining reinforcement learning and LLMs. Andrej Karpathy mentions it in his new talk (~38 minutes in) [1], and Ilya Sutskever talks about it in a lecture at MIT (~29 minutes in) [2]. It would be a huge breakthrough to find a proper reward function to train LLMs in a reinforcement learning setup, and to train a model to solve math problems in a similar fashion to how AlphaGo use…
Thanks for the links, very interesting. Wonder how a "self-play" equivalent would look like for LLMs, since they have no easy criterion to evaluate how well they are doing like in Go (as mentioned in the videos).
While this would still rely on a knowledge base in the LLM, I would imagine it could simplify the effort required to train reinforcement learning models, while widening the domains it could apply to.
I feel very comfortable saying, as a mathematician, that the ability to solve grade school maths problems would not be at all a predictor of ability to solve real mathematical problems at a research level. The reason LLMs fail at solving mathematical problems is because: 1) they are terrible at arithmetic, 2) they are terrible at algebra, but most importantly, 3) they are terrible at complex reasoning (more specifica…
it's always nice to see HN commenters with so much confidence in themselves that they feel they know a situation better than the people who are actually in the situation being discussed . Do you really believe that they don't have skilled people on staff? Do you really believe that your knowledge of what OpenAI is doing is a superset of the knowledge of the people who work at OpenAI? give me 0.1% of your confidence a…
I have a strong suspicion that this is a purposeful leak to hype up OpenAI's product. It wouldnt't be out of character for some of the people involved, and it's the kind of thing that rampant commercial (and valuation) focus can bring with it.