71–80 of 97 posts
Earlier quoted context omitted.
Do you hallucinate as a kid?
Kids definitely do this. They fill in blanks/context with assumptions, resulting in all sorts of silly responses, for topics of sparse knowledge/certainty. They're not lying, because they think it's true. Sometimes the gap filling is wrong, but usually downright brilliant, within the context of their knowledge.
Earlier quoted context omitted.
It's about the complexity of the task. Front end apps tend do be much less complex and boilerplate-y than backends, hence AI tends to work better.
I disagree - having worked on backends most of the time, I find modern frontend much more complex (and difficult to test) than pure backend. When I say modern frontend - its mostly React, state management like Redux, Zustand, Router framework like React Router, a CSS framework like Tailwind and component framework like Shadcn. Not to mention different versions of React, different ways of managing state, animation/tra…
AI is the best at adding standard things into standard boilerplate situations, all those frameworks just makes it easier for AI. They also make it easier for humans once you know them and have seen examples, that is why they exist, once you know those frontend is not hard.
Earlier quoted context omitted.
There was a well-publicised "Claude plays Pokémon" stream where Claude failed to complete Pokemon Blue in spectacular fashion, despite weeks of trying. I think only a very gullible person would assume that future LLMs didn't specifically bake this into their training, as they do for popular benchmarks or for penguins riding a bike.
While it is true that model makers are increasingly trying to game benchmarks, it's also true that benchmark-chasing is lowering model quality. GPT 5, 5.1 and 5.2 have been nearly universally panned by almost every class of user, despite being a benchmark monster. In fact, the more OpenAI tries to benchmark-max, the worse their models seem to get.
Earlier quoted context omitted.
Yes, at least to some extent. The author mentions that the base model knows the answer to the switch puzzle but does not execute it properly here. "It is worth noting that the instruction to "ignore internal knowledge" played a role here. In cases like the shutters puzzle, the model did seem to suppress its training data. I verified this by chatting with the model separately on AI Studio; when asked directly multiple…
My issue with this is that the LLM could just be roleplaying that it doesn't know.
Instead, what can happen is that, like a human, the model (hopefully) disregards the instruction, making it carry (close to) zero weight.
The baked-in assumptions observation is basically the opposite of the impression I get after watching Gemini 3's CoT. With the maximum reasoning effort it's able to break out of the wrong route by rethinking the strategy. For example I gave it an onion address without the .onion part, and told it to figure out what this string means. All reasoning models including Gemini 2.5 and 3 assume it's a puzzle or a cipher (be…
I've had some weird "thinking outside the box" behavior like this. I once asked 3 Pro what Ozzy Osbourne is up to. The CoT was a journey, I can tell you! It's not in its training data that he actually passed away. It did know he was planning a tour though. It had a real struggle trying to consolidate "suspicious search results" and even questioned whether it was fake news, or running against a simulation!, determinin…
Earlier quoted context omitted.
If they trained the model to respond to that, then it can respond to that, otherwise it can't necessarily.
I think you got a point here. These companies are injecting a lot of datasets every day into it.
Is this baked into how the models are built? A model outputs a bunch of tokens, then reads them back and treats them as the existing "state" which has to be built on. So if the model has earlier said (or acted like) a given assumption is true, then it is going to assume "oh, I said that, it must be the case". Presumably one reason that hacks like "Wait..." exist is to work around this problem.
1.88 billion tokens * $12 / 1M tokens (output) suggests a total cost of $22,560 to solve the game with Gemini 3 Pro?
True though I bet the $200 a month plan could do it, maybe a few extra days of downtime when quota was maxed
And yeah, it's not the insanely priced AI Ultra plan, but if there are any hard limits on Gemini Pro usage I haven't found them. I have played a lot with really long Antigravity sessions to try to figure out what this thing is good for, and it seems like it will pretty much sit there and run all day. (And I can't really blame anyone for still remaining mad about AI to be completely honest, but the technology is too neat by this point to just completely ignore it.)
Seeing as Google is still giving away a bunch of free access, I'm guessing they're still in the ultra-cash-burning phase of things. My hope (hopium, realistically) is that by the time all of the cash burning is over, there will be open-weight local models that are striking near where Gemini 3 Pro strikes today. It doesn't have to be as good, getting nearby on hardware consumers can afford would be awesome.
But I'm not holding my breath, so let's hope the cash burning continues for a few years.
(There is, of course, the other way to look at it, which is that looking at the pricing per token may not tell the whole story. Given that Google is running their own data centers, it's possible the economic proposition isn't as bad as it looks. OTOH, it's also possible it is worse than it looks, if they happen to be selling tokens at a loss... but I quite doubt it, given they are currently SOTA and can charge a premium.)
1.88 billion tokens * $12 / 1M tokens (output) suggests a total cost of $22,560 to solve the game with Gemini 3 Pro?
Did the streamer get subsidized by Google?
(The stream isn't run by Google themselves, is it?)
Earlier quoted context omitted.
Kids definitely do this. They fill in blanks/context with assumptions, resulting in all sorts of silly responses, for topics of sparse knowledge/certainty. They're not lying, because they think it's true. Sometimes the gap filling is wrong, but usually downright brilliant, within the context of their knowledge.
Are you sure there is an age limit for that kind of behavior in humans?