221–224 of 224 posts
Earlier quoted context omitted.
I mean dogs can learn a simple sign language?
Can the dogs sign back? Even dogs that learn to press buttons are mostly just pressing them to get treats. They don't ask questions, and it's not really a conversation.
Earlier quoted context omitted.
I just got this from codex yesterday: "I wasn’t able to finish; no changes were shipped." And it's not the first time.
Have you threatened it with a 2 in the next round of performance reviews?
Earlier quoted context omitted.
Most amazing is asking any of the models to draw an 11-sided polygon and number the edges.
I asked gpt5, and it worked really well with a correct result. Did you expect it to fail?
Earlier quoted context omitted.
> I don't think there's evidence that this issue would persist after continuing to scale models to be larger and doing more RL And how much larger do we need to make the models? 2x? 3x? 10x? 100x? How large do they need to get before scaling-up somehow solves everything? Because: 2x larger, means 2x more memory and compute required. Double the cost or half the capacity. Would people still pay for this tech if it doub…
Scaling applies to multiple dimensions simultaneously over time. A frontier model today could be replicated a year later with a model half the size, with a quarter of the FLOPS, etc. I don’t know the real numbers for optimization scaling, but you could check out NanoGPT speedrun [1] as an example. The best solution in the meantime is giving the LLM a harness that allows tool use like what coding agents have. I suspec…
Models "emergent capabilities" rely on encoding statistical information about text in their learnable params (weights and biases). Since we cannot compress information arbitrarily without loss, there is a lower bound on how few params we can have, before we lose information, and thus capabilities in the models.
So while it may be possible in some cases to get similar capabilities with a slightly smaller model, this development is limited and cannot go on for an arbitrary amount of time. It it were otherwise, we could eventually make a LLM on the level of GPT-3 happen in 1KB of space, and I think we can both agree that this isn't possible.
> giving the LLM a harness that allows tool use like what coding agents have
Given the awful performance of most coding "agents" on anything but the most trivial problems, I am not sure about that at all.