351–360 of 490 posts
Kimi K3 was an interesting model only a month ago, and now we're looking at the same performance for 1/20th of the price. Wild how fast this is advancing.
Real question: is there anybody that is both maintaining alpha-dev capability by keeping abreast of all these daily changes, while also reserving enough time to actually work? Seems like we've reached the event horizon of whether AI advances are worth paying attention to.
Earlier quoted context omitted.
> You don't win the stock market or make the deadliest drone by switching to the cheap model Really? How many times a small team has outperformed a much bigger one just because they were "doing it right"? I have been in software companies where most software produced was bad. Not just the code, the overall design everywhere. So... bad engineers with the most expensive model, or great engineers with cheaper models?
There are more than two options. What about great engineers with great models?
> You don't win the stock market or make the deadliest drone by switching to the cheap model
The question is not "can you win with the best model?", it is "can you not win without the best model?".
Earlier quoted context omitted.
That doesn’t make sense. It’s not like SOTA models are error free, yet we still use them. You use Fable 5 right? If that’s good enough for you now, why wouldn’t a Chinese model that’s as good as Fable 5 but at 10% the cost be good enough in 6 months?
I think we put up with Fable's occasional hiccups because there's nothing better at the moment. I use Claude Code semi-heavily for my small business, and the $100/mo I pay for that is a rounding error compared to the value it provides. If I can avoid spending an hour or two "massaging" the output from a lower-end model once, or it avoids introducing one load-bearing (sorry, couldn't resist) bug, then that's the entir…
Earlier quoted context omitted.
Except Fable won’t be costing $100 for enterprises that will be considering the Chinese models. If $100 Claud Max subscription works for you, then great. But you have to remember your pricing is subsidized by enterprises that pay hundreds of thousands of dollars each month, if not more, to Anthropic. For those companies, a Chinese model that can cut their AI spend from $1M/month to $200k suddenly seems attractive. An…
Didn’t deepseek recently announce prices will go up significantly? Right now the US dominates everyone else in actual chips in data centers. So even if deepseek etc tries to undercut, they’re very capacity limited.
It's that good. They are far from capacity limited, and even if they were, you can rent a single MI300X from somewhere like Hot Aisle and get more tk/s than you'll be able to use.
Earlier quoted context omitted.
If what you're saying is true and accurate, then US-based AI labs are in big trouble. The only saving grace might be some sort of a 'national security' proclamation banning the use of state-of-the-art Chinese (and non-US) models across US federal and state governments and large enterprises (especially ones with federal government contracts), but even still, US AI labs will probably lose out massively on international…
It is amazing how fast it happened. Right now one of my main projects is fully running on DeepSeek flash. My reason was that I was blocked by both of the main US AI labs from working on it because it involves viruses. DeepSeek flash has been killing it since I switched it on, completing the first phase of the project and setting up an iteration in another application space. It isn't the most brilliant model, but it i…
Many devs who have never tried from either side build it all up in their head but it's almost always been a matter of thorough tedium, which LLMs are excellent at churning through, especially when there's api docs/headers/code comments.
If you have the space, try mirroring your port at the switch level and capturing every packet then making it go through them all to look for whatever. We have NSA at home lol
Earlier quoted context omitted.
> it's good enough to use it for (almost) everything which in your case is?
I've posted a few times about my project that's a collection of 30k-250k webapps that are served from a WebDAV server. The apps know how to write updated copies of themselves back to the server. My family uses it. I have gallery apps (yearbooks for each year are a lot of fun!) of us on trips and just living, an outlining app that's a mesh of Workflowy and Org Mode (it's called Fluxtral), a markdown-backed app (it use…
Earlier quoted context omitted.
> it's good enough to use it for (almost) everything which in your case is?
I've posted a few times about my project that's a collection of 30k-250k webapps that are served from a WebDAV server. The apps know how to write updated copies of themselves back to the server. My family uses it. I have gallery apps (yearbooks for each year are a lot of fun!) of us on trips and just living, an outlining app that's a mesh of Workflowy and Org Mode (it's called Fluxtral), a markdown-backed app (it use…
With unsloth's Q3_S quant + kyuz0 'llama-vulkan-radv-performance' toolbox, I am getting 280+tps (batch and ubatch at 2048) for PP and 18+tps for TG. I really only need 256k context so it all fits.
If I go down to the Q3_XXS quant + dpsark + 'llama-vulkan-radv-performance', I can get about the same PP and 25+tps for TG with draft set to 2 or 3. Fits about the same as above.
Edit: I did notice the 25+tps quickly degrades down to 20+ after the first few hundred tokens.
Earlier quoted context omitted.
I think we put up with Fable's occasional hiccups because there's nothing better at the moment. I use Claude Code semi-heavily for my small business, and the $100/mo I pay for that is a rounding error compared to the value it provides. If I can avoid spending an hour or two "massaging" the output from a lower-end model once, or it avoids introducing one load-bearing (sorry, couldn't resist) bug, then that's the entir…
Except Fable won’t be costing $100 for enterprises that will be considering the Chinese models. If $100 Claud Max subscription works for you, then great. But you have to remember your pricing is subsidized by enterprises that pay hundreds of thousands of dollars each month, if not more, to Anthropic. For those companies, a Chinese model that can cut their AI spend from $1M/month to $200k suddenly seems attractive. An…
Think about it this way.
Let’s say you could buy an LLM that gets things right 98% of the time. But there’s another LLM that’s 100x the price but gets things right 99.9% of the time. To the lay person this sounds trivial but to a serious business this intelligence gap could represent millions, or billions of dollars.