I want to see more text-free foundation models
60% Fable cost cut by converting code to images and having the model OCR it
11–20 of 115 posts
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#12This seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?
It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
Of course it isn't
A text encoding uses 8bits per character on average, tokenization further compresses that
An image font would be 25 bits if 5x5, and most fonts are 12 pixels high
Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#13That is hilarious and an amazing find.
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#14Earlier quoted context omitted.
It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)
You are wrong.
Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token.
DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1]
Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction.
[1] https://github.com/deepseek-ai/DeepSeek-OCR/blob/main/DeepSe...
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#15Earlier quoted context omitted.
It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)
[dead]
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#16seems really dumb and like it would need to violate basic information theory to work?
input tokens are cheaper than output tokens. seems like it would maybe reduce input tokens at the expense of many more output tokens if you're actually triggering OCR via thinking?
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#17This seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?
It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
Step back and think about it another way - ask which scenario is more likely:
Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years. That trick being to rasterize 8bit characters into 8x8 pixels in a big image. 60% in a market worth trillions of dollars.
or
Anthropic's marketing team arbitrarily prices tokens to drive growth, according to vibes and feelings, and didn't think they needed to price images on par with text in their rush to burn cash & drive growth. Some folks take advantage of the trick during the first few days of the model's availability before Anthopic corrects their pricing, to align more proportionally with actual compute costs.
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#18Re: 60% Fable cost cut by converting code to images and having the model OCR it
#19Earlier quoted context omitted.
Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)
You are wrong. Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token. DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1] Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction. [1] https://…
A token is probably not a single char, and an image is probably decomposed into tokens as well (and god knows how many tokens an image is decomposed into) which probably map to similar float-hungry vectors. Your counterargument could use a bit more flesh.
And we're talking about images of texts, not images that represent complex imagery such as a very detailed scene or what have you.
Re: 60% Fable cost cut by converting code to images and having the model OCR it
#20Earlier quoted context omitted.
It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.
Step back and think about it another way - ask which scenario is more likely: Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years. That trick being to rasterize 8bit characters into 8x8 pixels in a big image. 60% in a market worth trillions of dollars. or Anthropic's marketing team arbitrarily prices tokens to drive gr…
> Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years of multi-trillion dollar growth
DeepSeek published a pretty well circulated paper on exactly this many months ago. It just hasn’t been attempted and shared publicly, asa retrofit, AFAIK.
Also, it’s no free lunch, the readme indicates that this “use images” hack is lossy and reduces success rates alongside the reduced cost. Most labs would focus on success increases regardless of price.