Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can you save on LLM tokens using images instead of text?

pagewatch.ai

Post reply on HN
Print view View original

11–20 of 21 posts

Re: Can you save on LLM tokens using images instead of text?

#11

Post by pastor_williams »

bikeshaving wrote: post #6
estebarb wrote: post #5

Earlier quoted context omitted.

The CALM paper https://shaochenze.github.io/blog/2025/CALM/ says it is possible to compress 4 tokens in a single embedding, so... image = 4×256=1024 words > 1000 words. QED

2.4% relative error is not bad.

Reminds me of Babbage making allowance for meter.

"""

    ... it is said that he [Babbage] sent the following letter to Alfred, Lord Tennyson about a couplet in "The Vision of Sin":

         Every minute dies a man,
         Every minute one is born

    I need hardly point out to you that this calculation would tend to keep the sum total of the world's population in a state of perpetual equipoise, whereas it is a well-known fact that the said sum total is constantly on the increase. I would therefore take the liberty of suggesting that in the next edition of your excellent poem the erroneous calculation to which I refer should be corrected as follows:

         Every minute dies a man,
         And one and a sixteenth is born

    I may add that the exact figures are 1.167, but something must, of course, be conceded to the laws of metre.
"""

    Charles Babbage and his Calculating Engines

Re: Can you save on LLM tokens using images instead of text?

#13

Post by estebarb »

behnamoh wrote: post #8
estebarb wrote: post #5

Earlier quoted context omitted.

The CALM paper https://shaochenze.github.io/blog/2025/CALM/ says it is possible to compress 4 tokens in a single embedding, so... image = 4×256=1024 words > 1000 words. QED

how do you decompress all those 4 words from one token?

Not from one token, from one embedding. Text contains a low amount of information: it is possible to compress a few token embeddings into a single tiken embedding.

The how is variable. The calm paper seems to have used a MLP to compress from and ND input (N embeddings of size D) into a single D embedding and other for decompress them back

Re: Can you save on LLM tokens using images instead of text?

#14

Post by HarHarVeryFunny »

behnamoh wrote: post #8
estebarb wrote: post #5

Earlier quoted context omitted.

The CALM paper https://shaochenze.github.io/blog/2025/CALM/ says it is possible to compress 4 tokens in a single embedding, so... image = 4×256=1024 words > 1000 words. QED

how do you decompress all those 4 words from one token?

The mechanism would be prediction (learnt during training), not decompression.

It's the same as LLMs being able to "decode" Base64, or work with sub-word tokens for that matter, it just learns to predict that:

will be followed by (or preceded by) , or vice versa.

Re: Can you save on LLM tokens using images instead of text?

#15

Post by zahlman »

pastor_williams wrote: post #11
bikeshaving wrote: post #6

Earlier quoted context omitted.

2.4% relative error is not bad.

Reminds me of Babbage making allowance for meter. """ ... it is said that he [Babbage] sent the following letter to Alfred, Lord Tennyson about a couplet in "The Vision of Sin": Every minute dies a man, Every minute one is born I need hardly point out to you that this calculation would tend to keep the sum total of the world's population in a state of perpetual equipoise, whereas it is a well-known fact that the said…

Wouldn't "one and a sixth" be more accurate in both respects?

Re: Can you save on LLM tokens using images instead of text?

#16

Post by cma »

floodfx wrote: post #3

Why are completion tokens more with image prompts yet the text output was about the same?

Some multimodal models may have a hidden captioning step that may take completion tokens, others work on a fully native representation, and some do both I think.

Re: Can you save on LLM tokens using images instead of text?

#19

Post by cbhl »

pastor_williams wrote: post #11
bikeshaving wrote: post #6

Earlier quoted context omitted.

2.4% relative error is not bad.

Reminds me of Babbage making allowance for meter. """ ... it is said that he [Babbage] sent the following letter to Alfred, Lord Tennyson about a couplet in "The Vision of Sin": Every minute dies a man, Every minute one is born I need hardly point out to you that this calculation would tend to keep the sum total of the world's population in a state of perpetual equipoise, whereas it is a well-known fact that the said…

Shouldn't it be the other way around if the population is increasing? Every minute one is born = 1440 born/day, every minute and a sixteenth ~= 1335 dead/day for a net population increase of 105/day.

Re: Can you save on LLM tokens using images instead of text?

#20

Post by ashed96 »

weird-eye-issue wrote: post #18
ashed96 wrote: post #9

In my experience, LLMs tend to take noticeably longer to process images than text.

It has to get the image data first, basically just IO time before processing it

IIRC there's pre-processing (embedding/tokenization?) before feeding images to LLMs?

Hit this issue optimizing LLM request times. Ending up lowering image resolution. Lost some accuracy but could bear that.

Post reply on HN