Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Segmenting comic book frames

vrroom.github.io

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Segmenting comic book frames

#31

Post by matroid »

awdii wrote: post #30
matroid wrote: post #26

Earlier quoted context omitted.

The original blog post by Max Halford ( https://maxhalford.github.io/blog/comic-book-panel-segmentat... ) does exactly that. I love his approach because, unlike mine, it is simple, yet it goes a long way. I'd encourage you to check it out. Can you explain what you mean by motion comic generation? Sounds interesting!

Oh wow I'll check that out! We're basically working on tools for turning normal comics into semi-animated versions with things like puppet animation (similar to live 2d style), and stable diffusion models. A lot of the work for traditional approaches is segmentation and in painting, so that's what we're working on now, but we're also making tools for colorization, effects, and controllable animations!

All that sounds definitely a lot of fun. I was also working on colorization with SD ControlNet recently (https://vrroom.github.io/blog/2024/02/16/interactive-colorin...).

Re: Segmenting comic book frames

#33

Post by innocenat »

dylan604 wrote: post #19
doesnt_know wrote: post #17

Earlier quoted context omitted.

I'm more of a manga reader then a western comic one, but why would there be sentiment against this? In what way is the reading order of panels up to interpretation?

You've now focused too much on manga/comics. Stepping back a bit and looking at art more in general and using the provided example of looking at a painting, who are you to tell me where I should be looking. yeah yeah, i see the obvious thing your AI is trying to tell me where to look, but I'm looking at this less obvious thing that really strikes my fancy. maybe it's a blemish. maybe it's a unique brush stroke/techni…

But the point of this entire discussion is about manga/comics???

Re: Segmenting comic book frames

#34

Post by daemonologist »

makeitdouble wrote: post #14
jameshart wrote: post #7

Next AI challenge: try to infer the intended panel reading sequence, and the flow of speech bubbles/narrative. Would potentially be a useful augmentation to a digital comic book reader, refocusing from panel to panel in sequence. Not to mention making comic book content more accessible.

That sounds similar to having an AI explain where you should be looking in a painting, or where to pay attention in a movie. This might be genuinely useful as an accessibility feature, but I'd also see strong sentiment against it, potentially from the creators of the art.

I think aside from accessibility the main use case is for e-ink devices where the resolution isn't high enough to comfortably display the entire page at one and the refresh rate isn't high enough to pan/zoom quickly. With those limitations automatically "paging" through the panels probably makes for a much better reading experience (I haven't tried it myself).

Re: Segmenting comic book frames

#35

Post by daemonologist »

karaterobot wrote: post #2

I always thought a fun side gig (or even volunteer opportunity) would be defining panel areas for digital comics. Both because I'd get to read a lot of comics, and because a lot of comics I read are frustratingly bad at it, and it makes it much harder to enjoy them. Well, there goes AI taking our jerbs.

You could label a bunch for OP and contribute to your own demise :D (It sounds like there isn't a large existing dataset since the author had to label P&C for testing and generate data for training.)

Re: Segmenting comic book frames

#36

Post by dylan604 »

innocenat wrote: post #33
dylan604 wrote: post #19

Earlier quoted context omitted.

You've now focused too much on manga/comics. Stepping back a bit and looking at art more in general and using the provided example of looking at a painting, who are you to tell me where I should be looking. yeah yeah, i see the obvious thing your AI is trying to tell me where to look, but I'm looking at this less obvious thing that really strikes my fancy. maybe it's a blemish. maybe it's a unique brush stroke/techni…

But the point of this entire discussion is about manga/comics???

because that was the idea proposed on how to expand this concept to something else. that's how conversations evolve. someone says something that plants a seed for someone else, that then gets someone else thinking, that then, that then.

Re: Segmenting comic book frames

#38

Post by thih9 »

tehnub wrote: post #37

There’s a high quality, free comic reader app that doesn’t collect any data that implements this feature quite nicely: https://apps.apple.com/us/app/smart-comic-reader/id151117521...

> free comic reader app

Your link points to a paid app store entry. Is this the correct app?

Re: Segmenting comic book frames

#40

Post by probably_wrong »

LegitShady wrote: post #24

Segmenting panels is a graphic design element of storytelling that's part of the artist's job. Doing it programmatically ignores its actual nature in storytelling - establishing the amount and direction of the story beats, helping the reader understand the right reading order, and establishing the important elements on the page (Which beat is the most important, etc). It's an interesting tech but giving such an impor…

I half agree with you. Yes, the panel segmentation can be an element of the storytelling, but the truth is that lots of comics out there are not exploiting this ability, sticking to a square grid instead.

Yes, Bill Waterson used his fame to get out of the standard grid [1] but in a world where people read comics on their phones this technology is necessary. And if this stuff helps comic artists reach more readers, so be it. We can always hope that making simple tasks easy today will encourage artists to try harder things tomorrow.

[1] https://www.leaderonomics.com/articles/personal/bill-watters...

Post reply on HN