41–48 of 48 posts
Complaining about "bad charting" and posting a chart with y-axis that doesn't start at 0 is kinda weird.
Earlier quoted context omitted.
Eh no. Expecting a non-programmer to be able to generate this is very elitist. A static image is still the standard.
The author's job description is literally "Staff Software Engineer at OpenTeams. Dask maintainer."!
Earlier quoted context omitted.
I think the key there is “hardware you already own.” If you own a graphics card you bought for gaming or a laptop you bought for doing schoolwork there is $0 in cost of local AI tokens, because 100% of the cost was assigned to doing other things.
It's bad accounting to only look at marginal cost and ignore depreciating hardware asset costs.
This looks great! I also think speed should be part of the metric (i.e. how long does the model take to actually solve a task). For me, I prefer to run expensive models such as Sol on light reasoning, which usually gives me good answers with quick responses. For my style of coding (quick back-and-forths and corrections) it makes a big difference if a model comes back in 1-2 minutes compared to 5-10, and I am happy to…
Go to https://artificialanalysis.ai/ and scroll down to the second graph under “Speed & Latency”.
I think this is the most import graph on their page. I wish they would let us filter by intelligence, or pass rate, and then see this graph. This is the tradeoff that actually matters, cost/token or tok/s can be very misleading (take glm-5.3-flash as an example).
The complaint about not being able to switch between linear and log is valid, which is what I did for making a 3D speed/cost/quality frontier application for a recent meetup talk: https://www.williamangel.net/apps/model_performance.html Because speed is important, as the reasoning and hardware determine both cost and speed. it's a three dimensional tradeoff.
Earlier quoted context omitted.
I think the key there is “hardware you already own.” If you own a graphics card you bought for gaming or a laptop you bought for doing schoolwork there is $0 in cost of local AI tokens, because 100% of the cost was assigned to doing other things.
It's bad accounting to only look at marginal cost and ignore depreciating hardware asset costs.
Open Teams originally wanted to rent out open source developers to sponsors with Oliphant controlling everything. Now they pivoted to installing local LLMs (on what hardware exactly?). What will happen is that this will be the third consultancy with a lofty narrative after Enthought and Anaconda that Oliphant established. It is always bait-and-switch.