> neural networks, generally, are already fantastically useful in tools
Yes, I agree. You've highlighted the distinction I should have included of "prompt-based".
There's a vast gulf between these AI-researcher-based concept demos on one side and the NN-based features slowly getting implemented in real production tools. Like you, I've found it challenging to have constructive conversations about AI tooling with anyone not versed in real production workflows. To anyone with real industry experience it's obvious that so far these demos don't represent a threat to real production workflows or the skilled career professionals making a good living. It's not that they're not threatening, they're just threatening to replace a different type of job entirely. If you're one of the poor souls in an off-shore locale doing remote low-end piece-work like manning a stock photo/video clip farm or doing >$100 per piece gigs on Fivver - then, yeah, you should feel "threatened".
A meta-point I try to make in these conversations is that, at least so far, every actual paying creative job I've seen AI threaten are, IMHO, work I wouldn't wish on my worst enemy. These are low-paid entry-level sweatshop gigs and everyone doing them aspires to do something else as soon as they can. The two analogies I use are: 1) How the "threat" of robotics to jobs is actually playing out. So far, in industrial applications robots are replacing Amazon warehouse and manufacturing assembly line workers, literally today's equivalent of 1920s sweatshop work. Much like the heart-wrenching videos of children in Calcutta earning pennies sifting through junk piles for metal scraps, it'll be a better world when robots replace those jobs and humans have jobs designing, installing, programming and servicing the robots. Likewise in consumer robotics applications, so far, the robots in our house only vacuum the floors, change the cat litter box, and wash the dishes/clothes. Growing up my family spent a couple years living in Asia in the 1970s and we actually had a "wash ama" who came twice a week and washed our clothes manually with a washboard and a tub. Sounds quaint but in reality it was grueling labor. She was a lovely lady but I'm glad Maytag replaced that job.
The second analogy I often use is observing that self-driving cars are mainly a threat to Uber and Lyft drivers who often barely earn minimum wage and have no job security to start with. Career professionals actually working in real video and film production workflows feel as "threatened" by prompt-based AIs as Formula 1 drivers feel about self-driving cars. Why does current F1 champion Max Verstappen never get asked how he feels about AI self-driving cars coming for his job? :-) As you observed, anyone who understands the thousands of creative choices which comprise any shot in a quality film doesn't even see these prompt-based AI demos as relevant. Once you've heard a skilled cinematographer, colorist or director of photography spend over an hour deconstructing and debating the creative choices made in single shot or scene from a film, it's hard to even imagine these demos as a threat to that level of creative skill. But being able to crudely copy the traits of a composite of a thousand exemplars of the craft without understanding any of the interactions between those thousands of creative choices does make for impressive demos. Even though the fidelity of the crude copy is amazing, the fact is such shots are a random puree of a thousand different creative choices pulled from a thousand different great shots. That's the root of what unskilled people call the "AI-clip sheen". It won't be easy to eliminate from prompt-based clip generators because the nature of the NN is it doesn't understand the interactions of all those subtle creative choices it's aping. Mashing together one cinematographer's lens choice from one shot with another cinematographer's filter choice from another shot with a third cinematographer's film stock choice from another film and a colorist's palette from a fourth unrelated work and then training the output filter only against broad criteria like "looks good" or "like a high-quality art film" is not a strategy that, IMHO, will ever produce a true threat to skilled top-level production workflows.
At the same time, as you observed, NN's are already delivering tremendous value eliminating labor-intensive, repetitive manual production work like frame-by-frame rotoscoping and animation tweening, work no one actually in the industry is sorry to see humans being relieved of. While I think NN-based features in production tools will continue to expand the use cases they can assist, I'm not sure AI tools will ever completely replace high-skill production professionals. I've already mentioned the technical challenges based on how NNs work but even if these challenges are someday overcome, there's a more fundamental limitation which is economic. Although feature film, network-level television and high-end commercials have massive cultural reach and are huge industries, the overall economic value of the entire technical production workflow and related tooling isn't as large as most people imagine. From Panavision cameras, Zeiss film lenses and Adobe Premiere to Chapman camera cranes, Sachtler tripods and Kinoflow lights, it's a relatively small industry with no unicorn-level startups. Even assuming one could license all the necessary content and manually tag it, it's hard to imagine a viable business plan which justifies investing the hundreds of millions required to recruit top-level AI researchers, thousands of H100 GPUs, etc to create and train a tool that could really replace the top 1000 career production pros working in Hollywood. There are so many other markets AI can target which are potentially far more lucrative than high-end film and video production workflows. Even the handful of blockbuster Summer tent pole movies made each year that cost $200M to make only spend somewhere around $10M or $20M on production labor and tooling below the department head level. That's not enough money to fund AI replacement anytime in the foreseeable future. The total addressable market of high-end film and video production just isn't big enough to be an attractive target for investors to fund going after it.