Ok that's...just cheating. You can't take a benchmark like MMLU designed to test the performance of a single general language model and compare it to performance of a small specialized model designed to do well on MMLU.
Interfaze: A new model architecture built for high accuracy at scale
31–40 of 47 posts
Re: Interfaze: A new model architecture built for high accuracy at scale
#32Ok that's...just cheating. You can't take a benchmark like MMLU designed to test the performance of a single general language model and compare it to performance of a small specialized model designed to do well on MMLU.
It wasn't designed to do well on MMMLU, it's a general model designed for deterministic task like OCR, object detection, STT and more and a by product of that is great language abilities. It still has a transformer backbone giving great language skills while being good at other stuff.
See the full benchmark: https://interfaze.ai/leaderboards
Re: Interfaze: A new model architecture built for high accuracy at scale
#33Similar to a large action model?
Not directly, LAMs tend to be focused a lot on tool calling or trained for a set of specific action for example in the robotics field. Good tool calling might be a good by product of Interfaze but wasn't specifically trained for that use case.
The focus has been for deterministic outputs that require high accuracy. In situations where there is "one right answer"
Re: Interfaze: A new model architecture built for high accuracy at scale
#34does it handle source code extraction from images?
how do I run it locally?
Re: Interfaze: A new model architecture built for high accuracy at scale
#35Interesting approach! One question though: can the model do column detection?
The first OCR example returns output that does not detect the article columns - the bounding box is the entire first line.
Re: Interfaze: A new model architecture built for high accuracy at scale
#36Interesting approach! One question though: can the model do column detection? The first OCR example returns output that does not detect the article columns - the bounding box is the entire first line.
It can, you could try prompting the model to use object detection vision and text extraction, we realized when we purely extract text it does amazing at word/sentence level bounds since the text acts as the anchor. However, when you treat it as a object detection problem, it sees that chunk of text as a segment allowing you the extract it as one column bound. Give that a try.
Re: Interfaze: A new model architecture built for high accuracy at scale
#37does it handle source code extraction from images? how do I run it locally?
yeah it would treat it like an OCR task and extract it, you could prompt it to format it better with the code alignment.
We serve it though an API. Check out the docs: https://interfaze.ai/docs
It's free to gets started.
Re: Interfaze: A new model architecture built for high accuracy at scale
#38Amazing! I just tried the OCR capabilities with a photo of a DIN A4 page which was written with a typewriter. The image isn't the easiest to interpret. The text perspective is distorted because the page is part of a book and the page margin toward the spine of the book is very small. There are also many inline corrections due to typing errors while the page was written (backspace couldn't erase characters back then,…
New account created ~5 hours after this post, with a single comment specifically praising the model / product. I want to believe, but this sort of astroturfing isn't very encouraging.
Re: Interfaze: A new model architecture built for high accuracy at scale
#39Gave it a try for structured data extraction. Tested returning a JSON object from images.
The output was correct, and seemed deterministic, although I ran it only 2-3 times on the same image.
Main problem is response time: it took about 20-25 seconds for a simple structure of 5 fields. As such unusable at scale, let alone "real time" processing.
Other problem is cost, it is considerably more expensive than more established models for the same document, like flash-light.
Shame, the architecture is very interesting.
Re: Interfaze: A new model architecture built for high accuracy at scale
#40This is very cool, though I don't understand exactly what they've done here. Is it some kind of LLM with convolutional layers added? The graph doesn't exactly make it clear but it describes a pipeline that goes beyond the LLM, so the CNN could be a separate model there.
Here’s the academic paper behind it: https://arxiv.org/abs/2602.04101
Thanks. Well this is fascinating.
>Instead of a single transformer, we combine (i) a stack of heterogeneous DNNs paired with small language models as perception modules
It seems that we're reinventing the brain's organs one by one from first principles. (Though Transformer + Common Crawl unintentionally builds a whole bunch of them we don't even understand yet.)
I found some broader context and the whole thing is indeed very harness-shaped:
>Using Interfaze as a Tool Inside Your Agent
https://interfaze.ai/blog/using-interfaze-as-a-tool-inside-y...
Well, Harness is the wrong word here... "environment/tools the LLM interacts with" definitely fits though. Or "other organoid" to use the previous metaphor.