Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Fun sketch – Bring your sketches to life

funsketch.kigun.org

Post reply on HN
Print view View original

1–10 of 12 posts

Show HN: Fun sketch – Bring your sketches to life

#1

Post by mishu2 »

Hello HN,

I created a simple website which lets children (of all ages) sketch something and then animate it using AI. It's a small hobby project because I wanted to see what all the hype is about.

No account required (but because the sketch app allows image uploads and we all know we can't have nice things on the Internet, I will be moderating the submissions before showing them on the front page).

Stack: Excalidraw, Django, Postgres, Redis for caching, ComfyUI for the image to video workflow (Wan 2.2 14B I2V).

Let me know what you think. Merry Christmas!

Show HN: Fun sketch – Bring your sketches to life
funsketch.kigun.org

Re: Show HN: Fun sketch – Bring your sketches to life

#3

Post by mishu2 »

forgingahead wrote: post #2

Nice work! What hardware are you running ComfyUI on, specifically the Wan2.2 workflow? It must be expensive having a GPU running for a hobby project like this?

Thank you. I'm running it locally on a 4090 (24 GB).

I was running into OOM issues with Wan 2.2 before, but I found the latest version of ComfyUI can now run it (using about 17 GB of VRAM for what I assume is a quantized model?).

Re: Show HN: Fun sketch – Bring your sketches to life

#4

Post by fouc »

Pretty good, I've noticed the animation tends to veer off / hallucinate quite a lot near the end. It is clear that the model is not maintaining any awareness of the first image. I wonder if there's a way to keep the original model in the context, or add original image back in at the half way mark.

Re: Show HN: Fun sketch – Bring your sketches to life

#5

Post by VierScar »

fouc wrote: post #4

Pretty good, I've noticed the animation tends to veer off / hallucinate quite a lot near the end. It is clear that the model is not maintaining any awareness of the first image. I wonder if there's a way to keep the original model in the context, or add original image back in at the half way mark.

Could do something funky like convert it to grayscale, add a 4th "colour" channel and put the grayscale image it that

Re: Show HN: Fun sketch – Bring your sketches to life

#6

Post by mishu2 »

fouc wrote: post #4

Pretty good, I've noticed the animation tends to veer off / hallucinate quite a lot near the end. It is clear that the model is not maintaining any awareness of the first image. I wonder if there's a way to keep the original model in the context, or add original image back in at the half way mark.

Thank you. I've noticed that too, and also that it has a tendency to introduce garbled text when not given a prompt (or a short one).

This is using the default parameters for the ComfyUI workflow (including a negative prompt written in Chinese), so there is a lot of room for adjustments.

Re: Show HN: Fun sketch – Bring your sketches to life

#7

Post by fouc »

mishu2 wrote: post #6
fouc wrote: post #4

Pretty good, I've noticed the animation tends to veer off / hallucinate quite a lot near the end. It is clear that the model is not maintaining any awareness of the first image. I wonder if there's a way to keep the original model in the context, or add original image back in at the half way mark.

Thank you. I've noticed that too, and also that it has a tendency to introduce garbled text when not given a prompt (or a short one). This is using the default parameters for the ComfyUI workflow (including a negative prompt written in Chinese), so there is a lot of room for adjustments.

Oh I was wondering why some of the hallucinations introduced Chinese text/visuals, I'm guessing that might be due to the negative prompt.

Re: Show HN: Fun sketch – Bring your sketches to life

#8

Post by mishu2 »

fouc wrote: post #7
mishu2 wrote: post #6

Earlier quoted context omitted.

Thank you. I've noticed that too, and also that it has a tendency to introduce garbled text when not given a prompt (or a short one). This is using the default parameters for the ComfyUI workflow (including a negative prompt written in Chinese), so there is a lot of room for adjustments.

Oh I was wondering why some of the hallucinations introduced Chinese text/visuals, I'm guessing that might be due to the negative prompt.

I think the main reason is that the model has a lot of training material with Chinese text in it (I'm assuming, since the research group who released it is from China), but having the negative prompt in Chinese might also play a role.

What I've found interesting so far is that sometimes the image plays a big part in the final video, but other times it gets discarded almost immediately after the first few frames. It really depends on the prompt, so prompt engineering is (at least for this model) even more important than I expected. I'm now thinking of adding a 'system' positive prompt and appending the user prompt to it.

Re: Show HN: Fun sketch – Bring your sketches to life

#9

Post by mishu2 »

VierScar wrote: post #5
fouc wrote: post #4

Pretty good, I've noticed the animation tends to veer off / hallucinate quite a lot near the end. It is clear that the model is not maintaining any awareness of the first image. I wonder if there's a way to keep the original model in the context, or add original image back in at the half way mark.

Could do something funky like convert it to grayscale, add a 4th "colour" channel and put the grayscale image it that

I'm actually trying to reduce the 'funkyness', initially the idea was to start from a child's sketch and bring it to life (so kids can safely use it as part of an exhibit at an art festival) :)

There's a world of possibilities though, I hadn't even thought of combining color channels.

Re: Show HN: Fun sketch – Bring your sketches to life

#10

Post by mishu2 »

Thank you everyone for taking a look. The website had around 1,200 visitors and received over 90 sketches over the past 24 hours, and I'm happy to say I could approve almost all of them (all except 2 'rocket' sketches which were starting to look a bit dubious, but all in good fun).

The results look very interesting to me, and I think as a next step I will look into adjusting the prompts (both positive and negative). On the one hand, I'd like to keep the sketch/drawing look rather than going full photorealism as some of the videos do, on the other hand I don't want to restrict the users' creativity too much.

I plan to simplify this further and optimize it for tablets. Some friends said they'd like to have their kids try it out it and I like the idea, as by having to sketch it keeps the user in a more active role.

Please let me know if you have any other suggestions.

Post reply on HN