Text to Image Synthesis Using Thought Vectors
1–10 of 23 posts
Re: Text to Image Synthesis Using Thought Vectors
#2I can see this being useful for police sketch artists.
Re: Text to Image Synthesis Using Thought Vectors
#3This is lovely. As a lazy programmer, I would appreciate this as a web service. Instead of googling for an image to steal as placeholder art, I could request a uniquely generated image.
Re: Text to Image Synthesis Using Thought Vectors
#4I can see this being useful for police sketch artists.
There is still a long way to go, to be able to do that. The model currently generates 64 X 64 pictures and is trained on a very specific flowers image dataset. Nevertheless, it would be a great idea to experiment with such a dataset (of sketches and descriptions) if available.
Re: Text to Image Synthesis Using Thought Vectors
#5It's a little tricky getting this to work because you need two separate models working together, but I tried it out. Here's some of the samples I generated:
https://imgur.com/oZ4wzdC some definite weaknesses in the natural language embedding
https://imgur.com/MAupphr roses in general don't seem to work well. must not have been many in the dataset
You can see that it works better than one would expect, but there are definitely limits to the understanding. The flower and COCO datasets are, ultimately, not that big. What would be exciting is if you could train it on some extremely large and well-annotated dataset like Danbooru.
Re: Text to Image Synthesis Using Thought Vectors
#6I can see this being useful for police sketch artists.
A GAN would be helpful more for its latent space abilities than for any text-to-image capability. If you look at the GAN papers, you can see that they demonstrate how you can warp an image across multiple semantic dimensions like adding/remove eyeglasses, darkening hair, turning frowns into smiles etc. This would be great for sketches because you can have the victim walk through the latent space by recognizing which version looks more like the attacker, since people can recognize faces far better than they can verbalize detailed descriptions.
Re: Text to Image Synthesis Using Thought Vectors
#7It's a little tricky getting this to work because you need two separate models working together, but I tried it out. Here's some of the samples I generated: https://imgur.com/Uwp1wfu https://imgur.com/yuW9Yre https://imgur.com/oZ4wzdC some definite weaknesses in the natural language embedding https://imgur.com/MAupphr roses in general don't seem to work well. must not have been many in the dataset You can see that it…
One possible improvement can be training the text embeddings along with the entire model (Instead of using the pretrained embeddings like skip-thought-vectors). It is on my to-do list, I'll try it out.
Re: Text to Image Synthesis Using Thought Vectors
#8It would be cool to implement text to pizza image synthesis.
Re: Text to Image Synthesis Using Thought Vectors
#9This is lovely. As a lazy programmer, I would appreciate this as a web service. Instead of googling for an image to steal as placeholder art, I could request a uniquely generated image.
I think few have grasped how much of future output will be machine generated from existing work, and yet rather than violate copyright, almost be a necessity to ensure a copyright somewhere is not broken.
Re: Text to Image Synthesis Using Thought Vectors
#10I can see this being useful for police sketch artists.
Probably by the time it gets good enough to be useful to them it will be good enough to replace them.