Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

60% Fable cost cut by converting code to images and having the model OCR it

github.com

Post reply on HN
Print view View original

101–110 of 115 posts

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#101

Post by Salgat »

ricardobeat wrote: post #4
genxy wrote: post #3

This seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?

It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.

That's not what is happening. Claude isn't charging for the tokens it generates from the OCR on its side, but it's still processing the same number of tokens as if you had sent the text, just with the extra step of OCR on Claude's side. This is 100% a loophole that's burning extra resources.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#102

Post by razodactyl »

sebmellen wrote: post #31
aabhay wrote: post #5

Ahhh my eyes the vibe coded readme

It’s so painful to read the LLM-compressed explanations. I can’t exactly identify what it is, but it’s an immediate tell and literally requires twice the effort to comprehend. For example: > Honest caveat, visible in the clip: the pxpipe arm answered the count first and needed one follow-up nudge to also print the ledger balance in the requested one-line format; the plain arm followed the format on the first try. Leg…

Concise responses come from people with writing skills who can get to the point without adding more words to sound smart.

"Elegant prose instantiated through remarkably tailored execution of written word may allow an author's desired intention to flow in a certain way to achieve a precise effect whilst simultaneously allowing said author to sound of much higher mind and thought to the reader." - I probably butchered it but my point is that AI slop seems to be the average of the outputs.

Images that look similar to others because they're average of all the current outputs. Same with music and video. We're noticing when something is AI because it has this signature that's average to other outputs.

Original content is crafted even though inspired by other works.

We're at a weird point where AI is capable but constrained.

As compute increases and AI becomes more personalised I feel the current implosion will explode again into variety.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#103

Post by npn »

it is funny because nobody ever bother points out that they overcharge you for text input token price.

sure it was pretty resource intensity a few years before, but with turbo quant, sparse attention and various techniques, plus the advancing of hardware (dedicated prefill machine, memory pool for kv caching) the cost should be drastically reduced, and yet they still keep the same cost formula.

I can't help but laugh whenever someone proudly share how many billion input tokens they spent in their code sections and how much they saved with the subscription, meanwhile it is pretty much just electricity cost for the providers.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#104

Post by sachamorard »

__hugues wrote: post #16

seems really dumb and like it would need to violate basic information theory to work? input tokens are cheaper than output tokens. seems like it would maybe reduce input tokens at the expense of many more output tokens if you're actually triggering OCR via thinking?

It's far from being a foolish idea, and it seems to me that the project correctly documents its own limitations. It's rare to see a README list so precisely where the tool falls short. Then, yes, input tokens are cheaper than output. But when it comes to find ways to get reduce costs, you have to explore all the options.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#105

Post by nazgul17 »

hn_throwaway_99 wrote: post #30
aabhay wrote: post #10

In Gemini at least, if you look at how they process PDFs, they do an OCR and then feed the text + image to the model, without charging you for the text tokens (I believe). So my guess is that Claude’s backend is doing the same — so this hack is probably more of a loophole in token accounting that might get closed if Claude is doing what Gemini does

This is really fascinating to me. I was reading this article and originally agreed with you, "I mean, under the covers it's got to be converting to text tokens at some point, so there is no way it's actually cheaper for Claude itself to execute." But then there is a comment below talking about how DeepSeek was able to get a huge improvement in compression by using visual tokens, https://news.ycombinator.com/item?id=4…

That was DeepSeek OCR, not a DeepSeek lineage LLM. If the idea is introduced in LLMs, then you're right. But Gemini is not doing that, not yet. This is something I literally discussed with Claude last week, but took its word for it.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#106

Post by troglodytetrain »

So, just be careful with this, it very likely is switching to other less capable model hence the cost reduction. So looks like Fable but isn’t. So you are doing extra work when you could just switch the model back to opus 4.8 instead.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#108

Post by ricardobeat »

Salgat wrote: post #101
ricardobeat wrote: post #4

Earlier quoted context omitted.

It’s not a loophole, it just happens that encoding information as optical tokens is much more efficient than text.

That's not what is happening. Claude isn't charging for the tokens it generates from the OCR on its side, but it's still processing the same number of tokens as if you had sent the text, just with the extra step of OCR on Claude's side. This is 100% a loophole that's burning extra resources.

There is no OCR, in the traditional sense, involved.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#109

Post by netsharc »

TZubiri wrote: post #38
netsharc wrote: post #21

Earlier quoted context omitted.

huh, what if the image encoding is 8 bits per R, G, B values of the pixel, then one can encode the same amount of text in less pixel dimensions (3 letters would need 1 pixel instead of three 12x12 pixels) The top line can be the OCR-able instruction on how to decode the rest of the image, and the rest of the image would be random-looking colourful palette. It might not even need to use 8 bits per character, since ANS…

then it's no longer an image, as the one in the github repo, you would be encoding the text as characters and sending it as an image. You can achieve this by changing the extension of an image file from .bmp to .txt Guys, not to be mean, but maybe chill with the state of the art research and go back to studying fundamentals.

If you want to be a know-it-all poseur, at least back it up with data.

Re: 60% Fable cost cut by converting code to images and having the model OCR it

#110

Post by 3abiton »

Klathmon wrote: post #61
geor9e wrote: post #23

Earlier quoted context omitted.

If the trick were genuinely useful, and was well circulated months ago, the resource-starved inference providers would have squeezed this trick dry already, instead of wasting 60% of their tokens, waiting for users to implement it themselves in 5 minutes of effort.

That's like saying quantization isn't real because the frontier labs aren't using it in their production inference. This is a lossy process, it produces worse results. It might be worth it for some situations, but applying it to everything would just be making your SOTA model worse

The "trick" is well documented in their Deepseek-OCR paper, that builds on plenty of other work. It's just not simple to just switch a commonly used LLM architecture to a new one, but I don't doubt most frontier labs are already experimenting with it. This by itself a very active field of research.
Post reply on HN