201–210 of 307 posts
Earlier quoted context omitted.
People who say that LLMs memorize stuff are just as clueless who assume that there's any reasoning happening. They generate statistically plausible answers (to simplify the answer) based on the training set and weights they have.
What if that’s all we’re doing, though?
Or we do it most of the time :)
Earlier quoted context omitted.
> I think veo3 proves that ai can generalize 2d and even 3d games It doesn't. And you said it yourself: > generating a video under prompt constraints is basically playing a game. No. It's neither generating a game (that people can play) nor is it playing a game (it's generating a video ). Since it's not a model of the world in any sense of the word, there are issues with even the most basic object permanenece. E.g. h…
It is still doing a great job for a few frames, you could keep it more anchored to the state of the game if you prompt it. Much like you can prompt coding agents to keep a log of all decisions previously made. Permanenece is excellent, it slips often but it mostly because it is not grounded to specific game state by the prompt or by the decision log.
Also, prompting doesn't work as you imply it does.
Earlier quoted context omitted.
I recall this not being true at all for Dota and Starcraft. I recall AlphaStar performed much better than the top non-pro players, but it couldn't consistently beat the pro players with the budget that Google was willing to spend, and I believe the same was true of Dota II (and there they were even playing a limited form of the game, with fewer heroes and without the hero choice part, I believe).
As I recall, the Starcraft ones heavily involved being able to exploit the computer's advantage in "twitch" speed over any human, it's just a slightly more complicated way of how any aim-bot enabled AI will always beat a human in an FPS, the game is designed to reward a certain amount of physical speed and accuracy. In other words, the Starcraft AIs that win do so by microing every single unit in the entire game at t…
Earlier quoted context omitted.
But the logic seems flawed. He has built an AI system that fails to do X. That does not mean there isn't an AI system that can do X. Especially considering that a lot is happening in AI, as you say. Anyway, Carmack knows a lot about optimizing computations on modern hardware. In practice, that happens to be also necessary for AI. However, it is not __sufficient__ for AI.
> That does not mean there isn't an AI system that can do X. You are holding the burden of proof here...
Earlier quoted context omitted.
Organic adaption and persistence of memory I would say are the two major advancements that need to happen. Human neural networks are dynamic, they change and rearrange, grow and sever. An LLM is fixed and relies on context, if you give it the right answer it won't "learn" that is the correct answer unless it is fed back into the system and trained over months. What if it's only the right answer for a limited period o…
Yes and: and forget.
I imagine humans are limited by the # of synapses we have so it's useful to forget but maybe machines can move the useless stuff to deep storage until it's dug out, in the same way certain things can trigger a deep memory in humans.
Earlier quoted context omitted.
I kinda think I'm more or less the same...OK maybe we have different definitions of "pattern matching".
It's Plato's cave: We train the models on what are basically shadows, and they learn how to pattern match the shadows. But the shadows are only depictions of the real world, and the LLMs never learn about that.
Man I can't wait for this '''''AI''''' stuff to blow over. The back and forth gets a bit exhausting.
But even I can see that this ""AI"" stuff is not going to blow over. That ship has sailed. Even if the current models get only marginal improvements, the momentum is unquestionably, inarguably there to make the adoption and productization 10x or even 100x wider than it is now. Robotics, automatization, self-driving, all kinds of kiosks, military applications (gathering and merging sensor data, controlling drone swarms, etc.)...
Just the amount of money (it's going to be trillions before the decade is over) and the amount of students in the field (basically all computer science degrees nowadays teach AI in some form) guarantees we're stuck with ""AI"" forever (at least until it kills us or merges with us)
Earlier quoted context omitted.
Just go and ask ChatGPT or Claude something that can't possibly be in its training set. Make something up. If it is only memorising answers then it will be impossible for it to get the correct result. A simple nonsense programming task would suffice. For example "write a Python function to erase every character from a string unless either of its adjacent characters are also adjacent to it in the alphabet. The string…
LLMs don't "memorize" concepts like humans do. They generate output based on token patterns in their training data. So instead of having to be trained on every possible problem, they can still generate output that solves it by referencing the most probable combination of tokens for the specified input tokens. To humans this seems like they're truly solving novel problems, but it's merely a trick of statistics. These…
Because they are. This is some crazy semantic denial. I should stop engaging with this nonsense.
We have AI that is kind of close to passing the Turing test and people still say it's not intelligent...
Earlier quoted context omitted.
Imagine a human had read every book/publication in every field of knowledge that mankind has ever produced AND couldn’t come up with anything entirely new. Hard to imagine.
My hypothesis of the mismatch is centered around "read" - I think that when you wrote it, and when others similarly think about that scenario, the surprise is because our version of "read" is the implied "read and internalized" or at bare minimum "read for comprehension" but as very best I can tell the LLM's version is "encoded tokens into vector space" and not "encoded into semantic graph" I welcome the hair-splitte…
A human with all that data, if it could fit in their brain, would likely come up with something interesting. Even then... I'm not entirely sure it's so simple. I'd wager most of us have enough knowledge in our brains today to come up with something if we applied ourselves, but ideas don't spontaneously appear just because the knowledge is there.
What if we take our AI models and force them to continuously try making connections between unlikely things? The novel stuff is likely in the parts that don't already have strong connections because research is lacking but could. But how would it evaluate what's interesting?
What John Carmack is exploring is pretty revealing. Train models to play 2D video games to a superhuman level, then ask them to play a level they have not seen before or another 2D video game they have not seen before. The transfer function is negative. So, in my definition, no intelligence has been developed, only expertise in a narrow set of tasks. It’s apparently much easier to scare the masses with visions of ASI…
Seeing comments here saying “this problem is already solved”, “he is just bad at this” etc. feels bad. He has given a long time to this problem by now. He is trying to solve this to advance the field. And needless to say, he is a legend in computer engineering or w/e you call it. It should be required to point to the “solution” and maybe how it works to say “he just sucks” or “this was solved before”. IMO the problem…
This comment, with the exception of the random claim of "he is just bad at this", reads like a thinly veiled appeal to authority. I mean, you're complaining about people pointing out prior work, reviewing the approach, and benchmarking the output.
I'm not sure you are aware, but those items (bibliographical review, problem statement, proposal, comparison/benchmarks) are the very basic structure of an academic paper, which each and every single academic paper on any technical subject are required to present in order to be publishable.
I get that there must be a positive feedback element to it, but pay attention to your own claim: "He is trying to solve this to advance the field." How can you tell whether this really advances the field if you want to shield it from any review or comparison? Otherwise what's the point? To go on and claim that ${RANDOM_CELEB} parachuted into a field and succeeded at first try where all so-called researchers and experts failed?
Lastly, "he is just bad at this". You know who is bad at research topics? Researchers specialized on said topic. Their job is to literally figure out something they don't know. Why do you think someone who just started is any different?