51–60 of 114 posts
Earlier quoted context omitted.
Give it an AI-bubble pop and these will be flooding the market.
no they won't , the bubble is a financial thing. the demand is real and not going away.
Earlier quoted context omitted.
At 830tok/s * 1 hour that's almost 3M tokens which is just $0.54 worth of tokens at Deepseeks current output price.
How is that economically viable? They are selling at a loss?
Definitely not. Inference is not as expensive to operate as many people seem to assume. The frontier labs are probably making a lot of money from selling tokens. It’s covering all of the R&D costs like salaries, collecting training material, and running the large training operations that costs a lot of money.
Earlier quoted context omitted.
> Well, if no body can say when it will pop, then can we really say it's a bubble and it's overvalued? Well, given the literal trillions being spent, the only ways this pays off are: 1. AI replaces a non-trivial fraction of human employees. 2. Someone builds a Culture Mind, and humans become (hopefully) pampered pets of AIs we don't understand. Seems unlikely, but it would arguably count as a payoff even if it made m…
Can we see some actual numbers, projections, models instead of vibes?
Interest alone, at assumed 5%, amounts to about $150 billions per year. That's probably higher than the combined AI revenue of the top 3 providers.
Earlier quoted context omitted.
Agentic workloads are somewhere around 1%/0.5%/98.5% input/output/cached tokens. Cached tokens are pretty much free for inference providers (if they implement sparse and compressed attention properly) and throughput for input tokens is much higher. Lets assume that you've got 2 million input tokens, 1 million output tokens and 98.5 million cached tokens to process. That would cost 2 * $0.14 + 1 * $0.28 + 98.5 * $0.00…
Your math is a bit funny if you're assuming the 1/0.5/98.5 ratios: you doubled input and output tokens but not cached. If you double cached tokens to match your original ratio it works out to around $1.11, and if you 10x the cached token cost it's around $6.08. Based on your $0.83 estimate, the margin isn't great. This is within shooting distance of "at cost" which is probably pretty close to what DeepSeek is operati…
I think the margins of DeepSeek may be a bit better than with this vibe-coded framework here, since they had the liberty of optimizing their models for their own hardware.
For DeepSeek V3, they claimed a cost profit margin of 545%: https://github.com/deepseek-ai/open-infra-index/blob/main/20...
At the time, open frameworks were not anywhere close to achieving that number. Not sure whether they caught up. The software wizards at DeepSeek are quite skilled.
Another headline of “model runs on x”, which usually means “let’s list how much you give up to run on x”. Dumbed down quantization? No. Full intended inference weights preserved, so far so good. Slow performance? No again. Looks like you could get over 150 tokens/second. Give up context window size? Yes. Original model is trained for and served at 1M, this is 256k. A very practical tradeoff though. Codex is in this r…
Strange that in the prior art they didn't list DwarfStar, as it is able to run the same model (probably quantized differently though) in less memory. Maybe the author isn't aware of it?
Earlier quoted context omitted.
Agentic workloads are somewhere around 1%/0.5%/98.5% input/output/cached tokens. Cached tokens are pretty much free for inference providers (if they implement sparse and compressed attention properly) and throughput for input tokens is much higher. Lets assume that you've got 2 million input tokens, 1 million output tokens and 98.5 million cached tokens to process. That would cost 2 * $0.14 + 1 * $0.28 + 98.5 * $0.00…
> should not discount that DeepSeek also gets paid in data, which is probably more valuable to them That's agentic feedback loops for training, right? Any more detail on this, such as how they actually tell whether that data is good or not? That seems like a very hard problem, and like the value of that data is low compared to just building their own, controlled RL gyms.
I am not sure how they are using the data for post-training, but there probably are ways to get signal out of it, e.g. sentiment analysis when the user begins cursing at the agent, or checking whether the user continued another session with the generated code, or started a new session with the same starting point as before, i.e. they git-stashed.
Generally, you can train on data that is quite bad (e.g. the entire internet). It will still work, but take much longer compared to clean data.
Earlier quoted context omitted.
Well, if no body can say when it will pop, then can we really say it's a bubble and it's overvalued? I can tell you it will pop in 10 years and when it pops, it will still be 20x bigger than in 2026. Does that even make any sense? People said AI bubble will pop soon in 2024 and that it was overvalued. Turns out, many AI stocks 10x, 20x since 2024. Actual usage has gone exponential as well. Anthropic revenue went from…
> Well, if no body can say when it will pop, then can we really say it's a bubble and it's overvalued? Well, given the literal trillions being spent, the only ways this pays off are: 1. AI replaces a non-trivial fraction of human employees. 2. Someone builds a Culture Mind, and humans become (hopefully) pampered pets of AIs we don't understand. Seems unlikely, but it would arguably count as a payoff even if it made m…
I mean, it's pretty clear that's going to happen. How could it not?
The only question is whether resources are optimally allocated at the moment to prepare for this. That seems unlikely at best. So yes, there is probably a bubble, and if so, then yes, it will pop, and then life will go on, with resources better allocated. Just like when the dot-com bubble popped.
Earlier quoted context omitted.
It’s available on demand from a few cloud providers. Seems like the cheapest is AMD Developer Cloud ( https://www.amd.com/en/developer/resources/cloud-access/amd-... ) powered by Digital Ocean at $1.99/hour. Edit: Now I think about it, this might be the cheapest way to run the DeepSeek V4 Flash 0731 on a dedicated inference server at original weights. I haven’t run mixed load benchmarks but I guess it’s possible to g…
At 830tok/s * 1 hour that's almost 3M tokens which is just $0.54 worth of tokens at Deepseeks current output price.
To be clear this project doesn’t aim to achieve the best inference economics per token. MI300X doesn’t have native MXFP4 so it’s not even the right platform for the model. That’s why very few deployment recipes are available.
It’s interesting to me because MI300X is quite accessible to a small team with budget for just 1-2 GPUs. DeepSeek V4 Flash otherwise wouldn’t even fit on 2x H100s.
We can run several coding agents during the day and batch inference jobs overnight and serve the entire team with guaranteed privacy, without compromising precision or speed.
In fact we found that many inference providers are quantising the weights or even KV cache, and due to the low prices they serve at massive batches, resulting in unstable throughput. I ran GSM8K as a quick validation test and this deployment is “better” than the OpenRouter endpoint in a statistically significant way (I wouldn’t name the provider here). I will run some follow up benchmarks and update the repo when I find some time.