Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SHOW-1 and Showrunner Agents in Multi-Agent Simulations

fablestudio.github.io

Post reply on HN
Print view View original

41–50 of 54 posts

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#41

Post by dooraven »

dom96 wrote: post #22

I skimmed the paper and didn't see an answer to this: how much of the video did the AI actually generate? how much of it was touched up by humans and how much of it was actually drawn/animated solely by humans?

Based on my understanding after skimming through the paper (and assistance with Claude), the AI did not directly generate any full video content for a South Park episode. It seems like this was how the AI was used:

- Custom diffusion models were trained on South Park character and background image datasets. These models could then generate new South Park-style characters and backgrounds.

- GPT-4 was used to generate dialogue for scenes, based on prompts about the overall episode premise and plot points.

- An "AI camera system" was mentioned for scene setup, but details were not provided on how much of the camera work it handled. Voice cloning was used to generate audio clips of the dialogue.

Note: this is just a skim of the paper, entirely possible I and Claude may have have missed something.

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#42

Post by tudorw »

I've got some 'AI' actors endlessly repeating a scripted 'show', part experiment, part entertainment, all weird, it runs on gtp4all so I can throw the same script at different models and see the variance and performance on a consistent creative task, you can interact with the story via chat, it 'remembers' previous interactions, sort of, interactions won't affect the outcome, that's scripted, but it will change what the actors talk about in each scene, it all runs off a simple yaml script that's meant to be 'easy' to edit. It runs TTS using either ElevenLabs for £££ or Coquai for free, and Bark when I get a GPU, Stability AI creates accompanying images, although they don't sync with the audio, they might at some point...as admin I can mess with max token length and some other settings without restarting which is nice, it runs pretty much 24/7 on a £250 pc, it's designed to run at four or five times the speed you see the 'dev' channel: https://twitch.tv/m88t

any questions you can drop them in the twitch chat... or here...

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#43

Post by thefourthchime »

codetrotter wrote: post #18

Was the intro generated as well, or lifted directly from the real show? There is a significant drop in believability as soon as the first scene of the generated show starts after the intro. Look at the first few seconds of the first scene. Here, the three boys are just standing statically in the hall while they are talking. Now take South Park Episode 1, Season 1, from 1997. https://www.southparkstudios.com/episodes/…

“The marvel is not that the bear dances well, but that the bear dances at all.”

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#45

Post by ca_tech »

dom96 wrote: post #22

I skimmed the paper and didn't see an answer to this: how much of the video did the AI actually generate? how much of it was touched up by humans and how much of it was actually drawn/animated solely by humans?

An engadget story demonstrates their simulation tool and discusses their "Simulation" venture a little bit more. It does look like the video is animated by AI using pre-generated scenes and characters. https://www.engadget.com/the-simulation-ai-put-me-in-a-south...

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#46

Post by huslage »

I believe this sort of system is one of the main reasons why WGA and SAG-AFTRA are striking. Even though this seeks to "augment" the writing process, it ultimately leads to not needing a writer at all. It also just requires an actor provide enough audio to generate a believable voice. All of the limitations to fully automated episodic generation are essentially temporary.

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#47

Post by rgbrgb »

djbusby wrote: post #31
madeofpalk wrote: post #23

Earlier quoted context omitted.

To be fair, we've always had startups pretending they have "AI" or "machine learning" but it was just mechanical turk or someone on upwork.

Ten Finger Automation or Wizard of Oz are the terms my entrepreneur/investor group uses.

pg used "man behind the curtain" (riff on Oz) for our real estate startup :)

Re: SHOW-1 and Showrunner Agents in Multi-Agent Simulations

#48

Post by Applejinx »

fnordpiglet wrote: post #36
Applejinx wrote: post #15

The interesting thing here is that Matt and Trey have codified intentionality as a primary writing tool, and whatever this is, doesn't mention that once. That's a pretty major thing to overlook, like a blind spot or achilles' heel of the AI devs. In fact, this is a blind spot of ALL AI that I've yet seen. Matt and Trey approach writing scenes with the following intent: if a scene can be described as following up a pr…

I’m not entirely sure I get it. You can literally prompt the LLM with “therefore” or “but,” and it’ll continue. You simply take the text generated prior and append “therefore,” and it’ll keep generating. If you want to lend more intentionality you can frame it with “therefore ” and it’ll continue generating with that intention. I’ve been up all night flying internationally but I might have misunderstood.

No, not really. The body of text GPT is trained on gets its own interpretation of 'but' and 'therefore'. If you feed it 'therefore' maybe it'll start writing legal documents. If you feed it 'but' maybe it'll just contradict itself, or express some triviality.

What you'd need for Matt and Trey style 'but' and 'therefore' is entire prompts being introduced in the background and switched out. Imagine thousands of words of prompt. 'but' means, fundamentally obstruct something about where your whole prompt is heading, like a screenwriter introducing a twist that must be resolved before the story can continue. 'therefore' means describe something that unfolds obviously as a result of all that's been introduced in the prompt.

These are not sentence-level issues, not output-level stuff. These are prompt level. More than that, they're prompt level with intentionality: you have to understand how a 'but' will fundamentally obstruct your prompt, how a 'therefore' will integrate both your original prompt and the obstruction.

Assume you have to coherently switch around your prompt introducing new fundamentals, and still have that make sense. It might get you rather formulaic results (but, Luke loses his mentor! therefore he must study and meditate and get to the final goal through his own transcendence!) but that just shows you it's working. That's how you get from a pile of arbitrary time-wasting, to a capital S Story.

From there on out, it's about which stories to tell, how far you can depart from the norms while still providing the intentionality and purpose, and what the purpose is :)

Post reply on HN