31–40 of 297 posts
I'd love to see a tokens per second metric. I always prioritize speed over raw intelligence for flash models.
This model might have a perfect speed:
for i in range(100):
print(random.choices(words))Does anyone actually uses these smaller models for coding? If so, how? I usually Opus everything. Is the play to plan/design/architect with a heavier model than delegate structured tasks to these smaller ones? Would appreciate to hear someone's opinion on having done and tested both paths.
It's so weird to me that the benchmarks remain so low, but the models are marketed as revolutionary. And if you say that low coding capabilities aren't a problem, say that to the token price hike and 'general use' model setup. Why not sell it as a math agent? Why do I have to set up 4 agents to check each others' work?
It’s about bang for buck. That high a score for 5B params is pretty good, nigh unbelievable a short while ago. It is my belief that smaller models will get better and better, and even cloud SOTA models will shrink. Yet another reason the current buildout will feel like the railroads.
That's what I'm betting on anyway.
Earlier quoted context omitted.
What is your evidence for this claim?
They say hill climbing https://microsoft.ai/news/building-a-hillclimbing-machine-la... Unless they specifically clarify that the testing and training benchmarks are completely separate, we have to assume they test on the same 'hill' the model climbs.