51–60 of 117 posts
Producing specific niche models for 100 year old industries that have mountains of data and warehouses full of folders will be the european take on AI. It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.
I expect the bitter lesson to continue to be bitter. Mistral must at least attempt to catch up to SOTA 6 months ago.
Earlier quoted context omitted.
I am working in Mistral robotics team. I confirm this is map-less. The only inputs are the text prompt and the front camera rgb image.
So, it has to rely on exact situational step by step commands? I'm wondering how one could conceivably deploy this in a useful way. Usually you'd need to mark areas on the map and then the robot knows where to go, A* is trivial around obstacles once you have that and a lidar. And lidars are an order of magnitude cheaper than something that can run an 8B VLA. One could maybe autogenerate these text planning commands,…
The advantage over traditional approaches is presumably flexibility. LIDAR isn't going to solve an instruction like "find the man with the pink shirt".
> achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments) I would like to know what it did the other 23.4% of the time!
Earlier quoted context omitted.
I expect the bitter lesson to continue to be bitter. Mistral must at least attempt to catch up to SOTA 6 months ago.
Do they really? "SOTA" is great for development and creating content but for industrial needs.... perhaps they are not really "SOTA"?
Robotics (using physics sims)
Cybersecurity (red team / blue team)
Math (using automated proof checkers)
Programming (using compilers)
For the record I think robotics is a totally logical place to use this training approach and this is very impressive. But if we zoom out and think about LLMs in general I’m not sure this inspires confidence in AGI arriving any time soon. I would also propose that this is a form of overfitting / training-test contamination.
Take cybersecurity for example. Through brute force techniques you will gradually memorize all of the possible exploits. So when fable breaks into a DoD network everyone is shocked but in reality it basically memorized all possible exploits including some zero day.
I’d be much more interested to see if fables performance is preserved as new exploits arise (NOT zero day - negative day meaning exploits that don’t exist yet). Would fable still find them? Or would they need to retrain it on the new software stack continuously in order to identify the zero days.
This is an important distinction that I have not seen made before.
This analysis by Toby Ord demonstrates why it’s a problem if frontier improvements are coming from reinforcement learning (brute force methods) from a purely computational perspective: https://www.tobyord.com/writing/inefficiency-of-reinforcemen...
It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able…
I am working in Mistral robotics team. I confirm this is map-less. The only inputs are the text prompt and the front camera rgb image.
Or, I don't know, make your own vacuums.
Earlier quoted context omitted.
The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
The General Models' business-model is also looking more weak every iteration. Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page". Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing…
If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.
Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?