21–30 of 48 posts
Why use electricity alone for open models? Surely you’d want to spread the cost of hardware over the period too?
I think the key there is “hardware you already own.” If you own a graphics card you bought for gaming or a laptop you bought for doing schoolwork there is $0 in cost of local AI tokens, because 100% of the cost was assigned to doing other things.
Because speed is important, as the reasoning and hardware determine both cost and speed. it's a three dimensional tradeoff.
Why use electricity alone for open models? Surely you’d want to spread the cost of hardware over the period too?
I think the key there is “hardware you already own.” If you own a graphics card you bought for gaming or a laptop you bought for doing schoolwork there is $0 in cost of local AI tokens, because 100% of the cost was assigned to doing other things.
Based on these costs I'd almost expect the amount of gaming graphics memory to go down over the next few years putting more stress on running those local models.
Earlier quoted context omitted.
to be fair, you don't need to start the y-axis at zero [0] but for some of the graphs where the lowest value is close to 0 the best practice is to do so there's a fun Excel artifact where it auto-selects the 'relevant' range with no adjustment for how proportionally close to 0 the values are - a professional researcher publishing to a journal should know better (and should be ridiculed for not incorporating best prac…
IMO in this case is mandatory to start from 0 because it alters the visual perception. Just look at the first chart: the distance between Fable 5.1 and Sol is <5%, but it looks like 25 or 30%.
And as the complexity of your score increases, the cutoff goes up. We can quibble about where your personal cutoff is, but it aint 0.
When I accessed the site, it showed the FBI badge says that the site was blocked and redirect to fbi.gov !! WTH?
Not seeing that, that's weird... Maybe a site sharing the same IP is blocked by the FBI through your ISP?
Complaining about "bad charting" and posting a chart with y-axis that doesn't start at 0 is kinda weird.
What is bad is starting at 0, showing an indicator of a gap, and suddenly starting at 30 or whatever after the gap.
Earlier quoted context omitted.
IMO in this case is mandatory to start from 0 because it alters the visual perception. Just look at the first chart: the distance between Fable 5.1 and Sol is <5%, but it looks like 25 or 30%.
I disagree. The y-axis is some arbitrary intelligence score that we use as a proxy for performance on whatever our specific task happens to be. So it doesn't matter if a model is a 0, 1, or 20 along this axis, they are all useless for the tasks I want. And as the complexity of your score increases, the cutoff goes up. We can quibble about where your personal cutoff is, but it aint 0.
But even if it was between 0 and Inf+, it still gives you a wrong perspective, especially if you are not paying attention, on model capabilities.