It was so fast that I didn't realise it had sent its response. Damn.
The path to ubiquitous AI (17k tokens/sec)
11–20 of 471 posts
Re: The path to ubiquitous AI (17k tokens/sec)
#12try here, I hate llms but this is crazy fast. https://chatjimmy.ai/
Re: The path to ubiquitous AI (17k tokens/sec)
#13This would be killer for exploring simultaneous thinking paths and council-style decision taking. Even with Qwen3-Coder-Next 80B if you could achieve a 10x speed, I'd buy one of those today. Can't wait to see if this is still possible with larger models than 8B.
Re: The path to ubiquitous AI (17k tokens/sec)
#14This is like microcontrollers, but for AI? Awesome! I want one for my electric guitar; and please add an AI TTS module...
Re: The path to ubiquitous AI (17k tokens/sec)
#15Jarring to see these other comments so blindly positive. Show me something at a model size 80GB+ or this feels like "positive results in mice"
Positive results in mice also known as being a promising proof of concept. At this point, anything which deflates the enormous bubble around GPUs, memory, etc, is a welcome remedy. A decent amount of efficient, "good enough" AI will change the market very considerably, adding a segment for people who don't need frontier models. I'd be surprised if they didn't end up releasing something a lot bigger than they have.
Re: The path to ubiquitous AI (17k tokens/sec)
#16It was so fast that I didn't realise it had sent its response. Damn.
Hurrah, its dumb answer to the now classic "the car wash is 100m away, should I drive or walk?" appeared very quickly.
Re: The path to ubiquitous AI (17k tokens/sec)
#17Jarring to see these other comments so blindly positive. Show me something at a model size 80GB+ or this feels like "positive results in mice"
There are a lot of problems solved by tiny models. The huge ones are fun for large programming tasks, exploration, analysis, etc. but there's a massive amount of processing This is great even if it can't ever run Opus. Many people will be extremely happy about something like Phi accessible at lightning speed.
Re: The path to ubiquitous AI (17k tokens/sec)
#18This would be killer for exploring simultaneous thinking paths and council-style decision taking. Even with Qwen3-Coder-Next 80B if you could achieve a 10x speed, I'd buy one of those today. Can't wait to see if this is still possible with larger models than 8B.
It uses 10 chips for 8B model. It’d need 80 chips for an 80b model.
Each chip is the size of an H100.
So 80 H100 to run at this speed. Can’t change the model after you manufacture the chips since it’s etched into silicon.
Re: The path to ubiquitous AI (17k tokens/sec)
#19I tried the trick question I saw here before, about the make 1000 with 9 8s and additions only
I know it's not a resonating model, but I keep pushing it and eventually it gave me this as part of it's output
888 + 88 + 88 + 8 + 8 = 1060, too high... 8888 + 8 = 10000, too high... 888 + 8 + 8 +ประก 8 = 1000,ประก
I googled the strange symbol, it seems to mean Set in thai?
Re: The path to ubiquitous AI (17k tokens/sec)
#20I wonder if this is the first step towards AI as an appliance rather than a subscription?