141–150 of 189 posts
Earlier quoted context omitted.
Definitely needs faster and cheaper models. Fast and cheap models could replace software in tons of situations. Imagine a vending machine or a mobile game or a word processor where basically all logic is implemented as a prompt to an llm. It would serve as the ultimate high level programming language.
Why on earth would you implement a vending machine using an LLM?
Earlier quoted context omitted.
I'm not sure it would be that hard to debug. Make sure you can reproduce the llm state (by storing the random seed for the session, or something like that) and then ask it "why did you just now give that customer mountain dew when they ordered sprite?"
> and then ask it "why did you just now give that customer mountain dew when they ordered sprite?" Worse than useless for debugging. An LLM can't think and doesn't have capabilities for self-reflection. It will just generate a plausible stream of tokens in reply that may or may not correspond to the real reason why.
Just think about it as a human employee. Can they always say why they did what they did? Often, but not always. Sometimes you will have to work to figure out the misunderstanding.
Earlier quoted context omitted.
Unfortunately the intuition and the math proofs so far suggest that autoregressive training is learning the joint distribution of probabilistic streams of tokens much better than diffision models do or will ever do. My intuitive take is that the conditional probability distribtion of decoder-only autoregressive models is at just the right level of complexity for probabilistic models to learn accurately enough. Intuit…
This is tremendously interesting! Could you point me to some literature? Especially regarding mathematical proofs of your intuition? I’d like to recalibrate my priors to align better with current research results.
In theory intuitively the smoothing distribution has access to all the information that the filtering distribution has and some additional information therefore has a minimum lower than the filtering distribution.
In practice, because the smoothing input space is much bigger, keeping the same number of parameters we may not reach a better score because with diffusion we are tackling a much harder problem (the whole problem), whereas with autoregressive models we are taking a shortcut which happens to probably be one that humans are probably biased too (communication evolved so that it can be serialized to be exchanged orally).
Earlier quoted context omitted.
AI field desperately needs smarter models - not faster models.
If you have much more speed in the available time, for an activity like coding, you could use that for iteration, writing more tests and satisfying them, especially if you can pair that with a concurrent test runner to provide feedback. I'm not sure the end result would be lower scoring/smartness than an LLM could achieve in the same duration.
It probably wouldn’t with current models. That’s exactly why I said we need smarter models - not more speed. Unless you want to “use that for iteration, writing more tests and satisfying them, especially if you can pair that with a concurrent test runner to provide feedback.” - I personally don’t.
Earlier quoted context omitted.
Definitely needs faster and cheaper models. Fast and cheap models could replace software in tons of situations. Imagine a vending machine or a mobile game or a word processor where basically all logic is implemented as a prompt to an llm. It would serve as the ultimate high level programming language.
Why on earth would you implement a vending machine using an LLM?
Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…
Is this really what people are doing these days?
It's nice to see a team doing something different. The cost[1] is US$1.00 per million output tokens and US$0.25 per million input tokens. By comparison, Gemini 2.5 Flash Preview charges US$0.15 per million tokens for text input and $0.60 (non-thinking) output[2]. Hmmm... at those prices they need to focus on markets where speed is especially important, eg high-frequency trading, transcription/translation services and…
I would be extremely hesitant to assume a direct relationship between pricing and cost. A behemoth like Google is very willing to take significant losses for years to grow market share. Back in 2014-2015 Uber often charged less than the Boston subway, but it always cost them MUCH more under the hood. AFAIK they're still not profitable. Chinese companies will be similarly eager for market share, but not everyone has t…
Not sure if I would tradeoff speed for accuracy. Yes, it's incredible boring to wait for the AI Agents in IDEs to finish their job. I get distracted and open YouTube. Once I gave a prompt so big and complex to Cline it spent 2 straight hours writing code. But after these 2 hours I spent 16 more tweaking and fixing all the stuff that wasn't working. I now realize I should have done things incrementally even when I hav…
You left an LLM to code for two hours and then were surprised when you had to spend a significant amount of time more cleaning up after it? Is this really what people are doing these days?
Earlier quoted context omitted.
Why on earth would you implement a vending machine using an LLM?
Because it's easy and cheap. Like how many products use a Raspberry Pi or ESP32 when an ATtiny would do.