AI Assisted Video Generation: Story of Shepherd Boy and the Wolf
1–4 of 4 posts
Re: AI Assisted Video Generation: Story of Shepherd Boy and the Wolf
#2HOW THE VIDEO WAS MADE ?
- The story was taken from internet.
- A 4 line summary of the story was generated using ChatGPT
- For each summary line, a corresponding image was generated using Stable Diffusion
- In each image (4 in our case), the important object were masked using MetaAI SAM
- I have developed a custom image to white-board animation code which converted the images to videos (This was the most time-consuming part of the development process).
- The audio was generated using gTTS (Google Text-to-Speech)
- The sub-titles were generated with the help of OpenAI whisper
- All the different audios, videos and subtitles were somehow synchronized using FFMPEG and OpenCV (This part gave a lot of pain... ALL HAIL FFMPEG)
LinkedIn: https://www.linkedin.com/in/yogendra-yatnalkar-2477b3148/
Re: AI Assisted Video Generation: Story of Shepherd Boy and the Wolf
#3HOW THE VIDEO WAS MADE ? - The story was taken from internet. - A 4 line summary of the story was generated using ChatGPT - For each summary line, a corresponding image was generated using Stable Diffusion - In each image (4 in our case), the important object were masked using MetaAI SAM - I have developed a custom image to white-board animation code which converted the images to videos (This was the most time-consum…
Open Source?
Re: AI Assisted Video Generation: Story of Shepherd Boy and the Wolf
#4HOW THE VIDEO WAS MADE ? - The story was taken from internet. - A 4 line summary of the story was generated using ChatGPT - For each summary line, a corresponding image was generated using Stable Diffusion - In each image (4 in our case), the important object were masked using MetaAI SAM - I have developed a custom image to white-board animation code which converted the images to videos (This was the most time-consum…
Open Source?
Hey yes: https://github.com/yogendra-yatnalkar/storyboard-ai
Jfyi: Sorry I just hacked up end-to-end stuff right now. I am in the process of cleaning the codebase and documenting it. Will do that in next 2-4 days.