Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using “underdrawings” for accurate text and numbers

samcollins.blog

Post reply on HN
Print view View original

61–70 of 140 posts

Re: Using “underdrawings” for accurate text and numbers

#61

Post by IdiotSavage »

> Transform this image into a photographed claymation diorama of assorted artisan chocolates and candies […] viewed from a low-angle

Side note: whenever I read prompts for image generation, I notice very specific details which the model obviously ignored. Here the chocolates / candies in the last two images look anything but artisanal. They look very "sterile" and mass-produced. The viewing angle is also not accurate.

Why do we even bother writing such elaborate prompts, when the model ignores most of it anyway?

Re: Using “underdrawings” for accurate text and numbers

#62

Post by p-e-w »

danpalmer wrote: post #40
mkbosmans wrote: post #35

Earlier quoted context omitted.

Nobody says they have no limitations. The question is are those limitation fundamental, i.e. can we expect improvement, say within a year.

When I talk about fundamental limitations, I mean limitations that can't be solved, even if they could be improved. We have improved hallucinations significantly, and yet it seems clear that they are inherent to the technology and so will always exist to some extent.

“Seems clear” based on what?

Re: Using “underdrawings” for accurate text and numbers

#63

Post by p-e-w »

coldtea wrote: post #53
p-e-w wrote: post #25

Earlier quoted context omitted.

> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.

> People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist Some limitations are not rigorously demonstrated to be fundamental, but continuously present from the first early LLMs yes. Shouldn't the burden of proof be on those who say it can be done? And some limitations are fundamental, and have been rigorously demonstrated, e.g.:…

That paper’s abstract doesn’t carry its title, to put it mildly.

Re: Using “underdrawings” for accurate text and numbers

#64

Post by p-e-w »

Marazan wrote: post #58
p-e-w wrote: post #25

Earlier quoted context omitted.

> due to fundamental limitations People keep throwing this phrase around in relation to LLMs, when not a single “fundamental limitation” has been rigorously demonstrated to exist, and many tasks that were claimed to be impossible for LLMs two years ago supposedly due to “fundamental limitations” (e.g. character counting or phonetics) are non-issues for them today even without tools.

If you remove the auxiliary tools and just leave the core LLM then strawberry still has an undefined number of `r`s in it.

That’s false. Larger LLMs learn token decompositions through their training, and in fact modern training pipelines are designed to occasionally produce uncommon tokenizations (including splitting words into individual characters) for this reason. Frontier models have no trouble spelling words even without tools. Even many mid-sized models can do that.

Re: Using “underdrawings” for accurate text and numbers

#66

Post by lacksjoian »

xigoi wrote: post #42
nine_k wrote: post #32

Earlier quoted context omitted.

Nobody asked it to!

If it’s asked to generate an image, it should to everything in its powers to make the image good.

LLMs have no concept of what makes the output "good". Or to put it another way, if the LLM generates an image with jumbled numbers it's because that was the most likely output, hence it was a "good" image according to its weights.

Re: Using “underdrawings” for accurate text and numbers

#67

Post by xigoi »

hirako2000 wrote: post #51
xigoi wrote: post #39

Earlier quoted context omitted.

Every decent human artist knows to draw a sketch before painting something.

Humans even have the creativity to come up with sketching. Models don't have intelligence, even less so creative thinking.

Exactly, that’s my point.

Re: Using “underdrawings” for accurate text and numbers

#68

Post by pegasus »

p-e-w wrote: post #62
danpalmer wrote: post #40

Earlier quoted context omitted.

When I talk about fundamental limitations, I mean limitations that can't be solved, even if they could be improved. We have improved hallucinations significantly, and yet it seems clear that they are inherent to the technology and so will always exist to some extent.

“Seems clear” based on what?

For one, based on continuously frustrated hopes (and promises!) that hallucinations will go away.

Re: Using “underdrawings” for accurate text and numbers

#69

Post by Sammi »

locknitpicker wrote: post #20
danpalmer wrote: post #6

I'm glad that we're making progress towards a deeper understanding of what LLMs are inherently good at and what they're inherently bad at (not to say incapable of doing, but stuff that is less likely to work due to fundamental limitations). There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Or asking an LLM to write you the SQL query for your data…

> There's similarity here with, for example, defining the architecture of software, but letting an LLM write the functions. Not so long ago, this was how early adopters of LLM coding assistants claimed was the right way to use them in coding tasks: prompt to draft the outline, and then prompt to implement each function. There were even a few posts in HN on blogposts showing off this approach with terms inspired in an…

In short, LLMs are pretty great at working at a single level of abstraction at a time.

You can go from the highest level and all the way down to the lowest level with LLMs, you just have to work at it iteratively one level at a time.

Re: Using “underdrawings” for accurate text and numbers

#70

Post by kilpikaarna »

p-e-w wrote: post #64
Marazan wrote: post #58

Earlier quoted context omitted.

If you remove the auxiliary tools and just leave the core LLM then strawberry still has an undefined number of `r`s in it.

That’s false. Larger LLMs learn token decompositions through their training, and in fact modern training pipelines are designed to occasionally produce uncommon tokenizations (including splitting words into individual characters) for this reason. Frontier models have no trouble spelling words even without tools. Even many mid-sized models can do that.

Wait, where can I learn more about this? I don't doubt that varying the tokenization during training improves results, but how does/would that enable token introspection?
Post reply on HN