411–420 of 512 posts
Earlier quoted context omitted.
They say they are using https://github.com/tile-ai/TileRT - persistent CUDA kernel - tiled processing with overlapping read/writes - model designed with specific constraints in mind
Excuse me, do aliens live among us? 17 commits, 99% Python and multiplying the speed of GLM, Deepseek V4, MiMO 2.5?
Earlier quoted context omitted.
I'm using Deepseek-v4-pro as my main model and this is sometimes pretty annoying, I have to do some easy boring task, think "I'll just leave the agent to do it and go take a nap", but it's already done writing the code before I even walk away from the computer
Do you mean Flash and not Pro? I haven't tried it personally, but according to OpenRouter, the fastest DeekSeep V4 Pro providers are only ~50tps. That's slower than Claude Opus. https://openrouter.ai/deepseek/deepseek-v4-pro?sort=throughp...
it is hard to understand what the actually meaningful innovations are here / what TileRT is bringing to the table. - dflash: new-ish but February is ancient by the standards of the pace of AI innovation lately, I guess applying it to a 1T model is new-ish in the sense that the dflash researchers don't have the hw budget to prove that out - persistent engine kernel: this is like CUDA 101 - warp specialization: I think…
Earlier quoted context omitted.
I’ve been playing around with groq and GPT OSS which they run at 1000 TPS (20B) or 800 TPS (120B) and the speed feels quite magical. I haven’t tried cerebras’ 3000 TPS yet but I did try the demo of that 15,000 TPS model whose name escapes me right now. I’m not sure if it makes a meaningful difference for my actual work, but it sure is amazing to watch it generate a screen full of text in the blink of an eye. I do thi…
https://chatjimmy.ai/ ?
The speed is incredible and fun to see, but the model is rather weak to the point where I’m not sure it’s particularly useful for most people.
Fast AI seems genuinely exciting and somewhat unsettling to me. Right now Claude is faster than me on some tasks but we’re at least close. I have a prompt to clean up a PR that’s been running for 1h now and I expect it to take another few. It’s hard to imagine how the workflow would look like if it was near-instant. On the one hand, it might be easier to focus. Some prompts take so long that I start to multitask and…
Giving directions and verifying its output? But my mental capacity is still limited. I can make way more prompts, than I can read code.
Tokens per seconds is the "Megapixels" of AI marketing!
Fast AI seems genuinely exciting and somewhat unsettling to me. Right now Claude is faster than me on some tasks but we’re at least close. I have a prompt to clean up a PR that’s been running for 1h now and I expect it to take another few. It’s hard to imagine how the workflow would look like if it was near-instant. On the one hand, it might be easier to focus. Some prompts take so long that I start to multitask and…
I'm using Deepseek-v4-pro as my main model and this is sometimes pretty annoying, I have to do some easy boring task, think "I'll just leave the agent to do it and go take a nap", but it's already done writing the code before I even walk away from the computer
the way software engineering works these days reminds me a lot of factory workers on production lines that just sit in front of a production line all day and take out faulty items and/or perform a single step in the production of goods.
Earlier quoted context omitted.
Definitely not, there's a ton of potential realtime use cases and high throughput/low TTFT is exactly what they need.
Of course, megapixels are also useful if you want to print large sizes.
Earlier quoted context omitted.
I'm using Deepseek-v4-pro as my main model and this is sometimes pretty annoying, I have to do some easy boring task, think "I'll just leave the agent to do it and go take a nap", but it's already done writing the code before I even walk away from the computer
Agent mania setting in It's also pretty funny sometimes how it gives weird future roadmap estimates ("part 2 - 3 weeks, part 3 - 2 months", etc.) and when you tell it to actually do those changes it's pretty much done in half an hour
those estimates are based on previous human estimates (the datasets it's been trained on).
unironically, when your comments will become part of a dataset, LLMs will likely get much better at estimating.
now that i think about it, all these writings about LLMs will give LLMs something much like meta-cognition.