Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

60% Fable cost cut by converting code to images and having the model OCR it

github.com

Post reply on HN
Print view View original

31–40 of 115 posts

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#31

Post by sebmellen »

aabhay wrote: post #5

Ahhh my eyes the vibe coded readme

It’s so painful to read the LLM-compressed explanations. I can’t exactly identify what it is, but it’s an immediate tell and literally requires twice the effort to comprehend.

For example:

> Honest caveat, visible in the clip: the pxpipe arm answered the count first and needed one follow-up nudge to also print the ledger balance in the requested one-line format; the plain arm followed the format on the first try. Legibility is solved on Fable — single-reply format compliance is the remaining rough edge.

If I reread this four times, I can sort of interpolate what happened, but it’s mostly pointless and confusing information.

In my experience all models do this to an extent, but Claude seems to be the worst at this. GPT 5.5 is a bit more terse but seems to compress more valuable information.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#32

Post by solenoid0937 »

geor9e wrote: post #23
vineyardmike wrote: post #20

Earlier quoted context omitted.

> Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years of multi-trillion dollar growth DeepSeek published a pretty well circulated paper on exactly this many months ago. It just hasn’t been attempted and shared publicly, asa retrofit, AFAIK. Also, it’s no free lunch, the readme indicates that this “use images” hack is l…

If the trick were genuinely useful, and was well circulated months ago, the resource-starved inference providers would have squeezed this trick dry already, instead of wasting 60% of their tokens, waiting for users to implement it themselves in 5 minutes of effort.

[flagged]

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#33

Post by Groxx »

legel wrote: post #14
TZubiri wrote: post #12

Earlier quoted context omitted.

Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)

You are wrong. Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token. DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1] Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction. [1] https://…

I kinda wonder if it's extracting usable context from 2D proximity between lines? Normal text input wouldn't have that kind of information (though it could, and it's arguably just a lookahead/behind of N characters on average).

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#34

Post by DANmode »

hn_throwaway_99 wrote: post #30
aabhay wrote: post #10

In Gemini at least, if you look at how they process PDFs, they do an OCR and then feed the text + image to the model, without charging you for the text tokens (I believe). So my guess is that Claude’s backend is doing the same — so this hack is probably more of a loophole in token accounting that might get closed if Claude is doing what Gemini does

This is really fascinating to me. I was reading this article and originally agreed with you, "I mean, under the covers it's got to be converting to text tokens at some point, so there is no way it's actually cheaper for Claude itself to execute." But then there is a comment below talking about how DeepSeek was able to get a huge improvement in compression by using visual tokens, https://news.ycombinator.com/item?id=4…

It wouldn’t, they’re subsidizing it for training.

Edit: didn’t realize this occurred on local models(!!),

this is smarter https://news.ycombinator.com/item?id=48779884

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#35

Post by [deleted] »

ricardobeat wrote: post #4
genxy wrote: post #3

This seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?

It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.

[deleted]

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#36

Post by hashmap »

sebmellen wrote: post #31
aabhay wrote: post #5

Ahhh my eyes the vibe coded readme

It’s so painful to read the LLM-compressed explanations. I can’t exactly identify what it is, but it’s an immediate tell and literally requires twice the effort to comprehend. For example: > Honest caveat, visible in the clip: the pxpipe arm answered the count first and needed one follow-up nudge to also print the ledger balance in the requested one-line format; the plain arm followed the format on the first try. Leg…

the foamy hedging makes me ill and hurts my eyes

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#37

Post by TZubiri »

legel wrote: post #14
TZubiri wrote: post #12

Earlier quoted context omitted.

Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)

You are wrong. Text tokens are high-dimensional vectors, not 8 bits per character. Every token has a deep embedding, e.g. 1024 float values per text token. DeepSeek-OCR proved 10x+ compression from visual embedding of text, which was a groundbreaking result. [1] Very cool to see OP's project hacking on this principle. It's still not lossless, as noted in the github, but is a promising research direction. [1] https://…

>Text tokens are high-dimensional vectors,

You are conflating tokens with embeddings.

Tokens fit in a single word, modern gpt uses a vocabulary with 200k possible values, which would fit into 18 bits.

Have a good one

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#38

Post by TZubiri »

netsharc wrote: post #21
TZubiri wrote: post #12

Earlier quoted context omitted.

Of course it isn't A text encoding uses 8bits per character on average, tokenization further compresses that An image font would be 25 bits if 5x5, and most fonts are 12 pixels high Of course it isn't efficient, this is a pricing inefficiency and a hack to exploit it (even the author describes it as an exploit)

huh, what if the image encoding is 8 bits per R, G, B values of the pixel, then one can encode the same amount of text in less pixel dimensions (3 letters would need 1 pixel instead of three 12x12 pixels) The top line can be the OCR-able instruction on how to decode the rest of the image, and the rest of the image would be random-looking colourful palette. It might not even need to use 8 bits per character, since ANS…

then it's no longer an image, as the one in the github repo, you would be encoding the text as characters and sending it as an image.

You can achieve this by changing the extension of an image file from .bmp to .txt

Guys, not to be mean, but maybe chill with the state of the art research and go back to studying fundamentals.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#39

Post by rightbyte »

calebkaiser wrote: post #24
geor9e wrote: post #17

Earlier quoted context omitted.

Step back and think about it another way - ask which scenario is more likely: Some random person discovered a 60% across the board gain in all LLMs, using an extremely simple trick that none of the labs noticed in all these years. That trick being to rasterize 8bit characters into 8x8 pixels in a big image. 60% in a market worth trillions of dollars. or Anthropic's marketing team arbitrarily prices tokens to drive gr…

Nah, optical compression is a thing. You see it in a lot of different areas in ML. In this case, the "trick" has been known for a while, and belongs to a whole world of compression research. But I think where you're maybe getting mixed up is in where that 60% gain is coming from. It's not a 60% percent reduction in cost for 100% of the same output. If you have a model and input text A, and you fix the seed etc. and r…

> a compression strategy

To me compression implies smaller size? However new line chars seems to be removed in the pic so I guess it could be expressed in fewer bytes than the original text with further compression ...

Post reply on HN