Like many AI things, it would have been considerably easier just to learn to edit images in GIMP or something. Instead of learning a valuable skill, you spent time working with a model that will be obsolete in a few months. Sunken cost fallacy, I guess.
Using “underdrawings” for accurate text and numbers
101–110 of 140 posts
Re: Using “underdrawings” for accurate text and numbers
#102> Transform this image into a photographed claymation diorama of assorted artisan chocolates and candies […] viewed from a low-angle Side note: whenever I read prompts for image generation, I notice very specific details which the model obviously ignored. Here the chocolates / candies in the last two images look anything but artisanal. They look very "sterile" and mass-produced. The viewing angle is also not accurate…
I loved the example where he requested ‘studio lighting’ and it put a bunch of studio lights in the picture.
Re: Using “underdrawings” for accurate text and numbers
#103The standard objection: if the LLM is supposedly intelligent, why can’t it figure out on its own that this two-step process would achieve a better result?
They are not, in fact, intelligent.
Re: Using “underdrawings” for accurate text and numbers
#104Earlier quoted context omitted.
The literal best public models still fail to count characters consistently in practice so I’m not sure what you mean. It’s literally a problem we’re still trying to solve at work
What's amazing is that they even can fairly reliably appear to count characters. I mean we're talking about systems that infer sequences not character counters or calculators. They are amazing in unrelated ways and we need to accept this so we can use them effectively.
I suspect character counting - counting small numbers in general in fact - is something that multimodal models will gradually learn through their visual capabilities. We have generative systems that are capable of generating an image of the word ‘strawberry’, and of counting how many strawberries are visible in an image; seems likely it’s possible for an LLM to ‘imagine’ what the word strawberry looks like and count the ‘Rs’ it can ‘see’.
Re: Using “underdrawings” for accurate text and numbers
#105I'm glad that we're making progress towards a deeper understanding of what LLMs are inherently good at and what they're inherently bad at (not to say incapable of doing, but stuff that is less likely to work due to fundamental limitations). There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Or asking an LLM to write you the SQL query for your data…
> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.
> character counting
The models now whaste a vast amount of useless neurons memorising the character count the entire English language so that people can ask how many r's are in strawberry and check a tickbox in a benchmark.
The architecture cannot efficiently or consistently represent counting letters in words. We should never have forced trained them to do it.
This goes for other more important "skills" that are unsuited to tranformer models.
Most models can now do decent arithmetics. But if you knew how it has encoded that ability in its neurons then you would never ever ever ever trust any arithmetic it ever outputs, even in seems to "know" it (unless it called a calculator MCP to achieve it).
There are fundamental limitations, but we're currently brute forcing ourselves through problems we could trivially solve with a different tool.
Re: Using “underdrawings” for accurate text and numbers
#106Earlier quoted context omitted.
Nobody says they have no limitations. The question is are those limitation fundamental, i.e. can we expect improvement, say within a year.
As a general architecture, an LLM also has limitations that can't be improved unless we switch to another, fundamentally different AI design that's non LLM based. There are also limitations due to maths and/or physics that aren't fixable under any design. Outside science fiction, there is no technology whose limitations are all fixable. Here's one: https://arxiv.org/abs/2401.11817?utm_source=chatgpt.com
Am I misreading that paper? They define hallucinations as anything other than the correct answer and prove that there are infinitely many questions an LLM can't answer correctly, but that's true of any architecture- there are infinitely many problems a team of geniuses with supercomputers can't answer. If an LLM can be made to reliably say "I don't know" when it doesn't, hallucinations are solved- they contend that this doesn't matter because you can keep drawing from your pile of infinite unanswerable questions and the LLM will either never answer or will make something up. Seems like a technically true result that isn't usefully true.
Re: Using “underdrawings” for accurate text and numbers
#107Earlier quoted context omitted.
I wonder how long it took to come up with all this? Because if I wanted a spiral of little "buttons" like the last one at the end (and they don't look very much like sweets) I'd be able to knock that out in Blender in an afternoon, and I'm not very good at Blender.
I think you're vastly overestimating the average persons ability to use Blender if you can do that in an afternoon; just figuring out how to place a colored cube and the camera probably takes an afternoon if you pick up Blender for the first time.
I'd just do it with Pillow.
Re: Using “underdrawings” for accurate text and numbers
#108Task like this, are the cause IA is eating memory and cpu prices.
Re: Using “underdrawings” for accurate text and numbers
#109Earlier quoted context omitted.
> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.
> character counting The models now whaste a vast amount of useless neurons memorising the character count the entire English language so that people can ask how many r's are in strawberry and check a tickbox in a benchmark. The architecture cannot efficiently or consistently represent counting letters in words. We should never have forced trained them to do it. This goes for other more important "skills" that are un…
> The models now whaste a vast amount of useless neurons memorising the character count the entire English language
No they don’t. They only need to know the character count for each token, and with typical vocabularies having around 250k entries, that’s an insignificant number for all but the tiniest LLMs.
Re: Using “underdrawings” for accurate text and numbers
#110Earlier quoted context omitted.
Character counting errors are a side effect of tokenization, which is a performance optimization. If we scaled the hardware big enough we could train on raw bytes and avoid it.
No, tokenization is not the only reason. A next-word predictor has fundamentally a hard time executing algorithms, even as simple as counting.
Counting is one of the algorithms that can be expressed by a RASP program, which transformers closely approximate.