Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node-based a/v composition: programs as graphs and graphs as compositional tools

github.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#11

Post by andybak »

ssalazar wrote: post #10

From a creative perspective the modular/node-based way of thinking has always been a niche, having seen many attempts such as this to place it into a broader model of creativity (including my own [1]). Its telling that the enduring musical instruments of the electronic age have been relatively fixed-function (Minimoog, Yamaha FM synths, 808/909) with constrained options for patchability, rather than modular synths hi…

Flow-based, node-based, visual - whatever you want to call it, it is programming and needs one to think like a programmer for anything but the most simple tasks.

In fact - it can be more complex than "actual" coding in some ways as the pallette of abstractions is often more limited - or the interface creates hoops to jump through that a text-based format wouldn't.

(I do like jumping between text and visual node-based representations however. Learnability and discoverability are an advantage of the latter. And it fosters a different style of tinkering.)

With regard to timelines - I can think of several ways a timeline would work with a graph. It can simply be another input for any exposed parameters. Or it can be a node in it's own right.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#12

Post by kousun12 »

ssalazar wrote: post #10

From a creative perspective the modular/node-based way of thinking has always been a niche, having seen many attempts such as this to place it into a broader model of creativity (including my own [1]). Its telling that the enduring musical instruments of the electronic age have been relatively fixed-function (Minimoog, Yamaha FM synths, 808/909) with constrained options for patchability, rather than modular synths hi…

Agree on most of this; most systems I've used so far feel a bit clunky (Houdini has by far been the most sophisticated system I've worked in), and my sense is that if it had more mindshare / attention, the paradigm could actually be pretty fluid. Using most node-based tools today (including eternal), I get the admonition that it's like using text edit to write python compared to a pro at using pycharm.

The timeline thing is really key. Houdini does it in a sort of clever way where "time" as a variable is exported globally to any node that wants to use it, and the timeline feature is a consequence of using the time variable in any of your functions. i.e. it's not jumping you to a point in the program's execution, but rather acting as a controller for all time-dependent functions. sort of like exporting a time uniform to graphics shader

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#13

Post by ssalazar »

kousun12 wrote: post #12
ssalazar wrote: post #10

From a creative perspective the modular/node-based way of thinking has always been a niche, having seen many attempts such as this to place it into a broader model of creativity (including my own [1]). Its telling that the enduring musical instruments of the electronic age have been relatively fixed-function (Minimoog, Yamaha FM synths, 808/909) with constrained options for patchability, rather than modular synths hi…

Agree on most of this; most systems I've used so far feel a bit clunky (Houdini has by far been the most sophisticated system I've worked in), and my sense is that if it had more mindshare / attention, the paradigm could actually be pretty fluid. Using most node-based tools today (including eternal), I get the admonition that it's like using text edit to write python compared to a pro at using pycharm. The timeline t…

Agreed- a lot of the issues people bring up with node-based programming have promising solutions, its just a matter of exploring the space and finding successful implementations.

As far as timeline, I also think the digital audio workstation model is interesting- modules export parameters that can be assigned to an automation channel and dragged/tweaked visually on some hierarchy of timelines.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#14

Post by mroche »

BubRoss wrote: post #8

He didn't mention a few other big ones with good demo versions: Nuke, Touch Designer and Digital Fusion (much more awkward, but free). Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical limitations. They all operate on a very limited set of data types (just images, images + geom…

> Digital Fusion (much more awkward, but free)

Small nitpick, only applies to Fusion 9 and below. Fusion 16 only comes in the Studio variant, otherwise you need to use DaVinci Resolve 16 to access the embedded Fusion for free.

> Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical limitations.

In the case of Nuke, the pragmatic workflow is a large part of it, but not the only one. When it comes to digital compositing, it is very easy to get up and running with layer based solutions quicker (such as Adobe products), but they hit a wall very quickly on memory usage, process duplication, and general difficulty of navigation and organization as the composite becomes more complex.

Nuke is in a different class of its own, being developed originally at Digital Domain (a VFX studio) before eventually sold to Foundry (then The Foundry). Nuke's success was a technical one, being far more memory aware and efficient (a scanline renderer and processor), as well as catering directly to the user base it served. It is still the only specialist compositing tool that has support for modern deep data, has proper handling and support for multi-layer/multi-part EXRs, and has a clean and proper handling of channels and layers throughout the comp. It has some modernization to undergo nowadays, but it is still the most trusted tool in VFX.

Software in computer graphics to note as being primarily node based that expose that directly to the user:

  - Autodesk Maya (is node-based)
  - SideFX Houdini (formerly Prism)
  - Foundry Nuke (originated from Digital Domain)
  - Foundry Katana (originated from Sony)
  - Substance Designer
  - Isotropix Clarisse BUiLDER
  - Natron
  - Blackmagic Design Fusion (an in-house tool to Eyeon Fusion -> Blackmagic Design)
Other tools that integrate node based workflows into their application:

  - Autodesk 3DS Max
  - Foundry Mari (originated from Weta Digital)
  - Maxon Cinema 4D
  - Blender (personally unsure of Blender and being nodal-based)
  - Blackmagic Design DaVinci Resolve
There are a smattering of others I haven't gotten the chance to use, from other imaging tools to those specializing in tracking (e.g. PfTrack) and in-house non-commercialized tools.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#15

Post by anigbrowl »

jcelerier wrote: post #9

If someone's interested in moving the node-based media software status quo, I've been developing ossia score for the past few years : https://ossia.io A big difference with other similar environments, is that the dataflow graph evolves dynamically based on a timeline which is useful for interactive shows, etc. I've recently been working on doing the same for shader-based stuff : https://twitter.com/jcelerie/status/12…

Intriguing. Very surprised I've never heard about this before, and refreshingly different from most other tools in this field. I look forward to trying it out.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#16

Post by BubRoss »

mroche wrote: post #14
BubRoss wrote: post #8

He didn't mention a few other big ones with good demo versions: Nuke, Touch Designer and Digital Fusion (much more awkward, but free). Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical limitations. They all operate on a very limited set of data types (just images, images + geom…

> Digital Fusion (much more awkward, but free) Small nitpick, only applies to Fusion 9 and below. Fusion 16 only comes in the Studio variant, otherwise you need to use DaVinci Resolve 16 to access the embedded Fusion for free. > Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical…

I wrote plugins for Nuke 12 years ago, I was just trying to give an interesting introduction to programmers.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#17

Post by PunksATawnyFill »

mroche wrote: post #14
BubRoss wrote: post #8

He didn't mention a few other big ones with good demo versions: Nuke, Touch Designer and Digital Fusion (much more awkward, but free). Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical limitations. They all operate on a very limited set of data types (just images, images + geom…

> Digital Fusion (much more awkward, but free) Small nitpick, only applies to Fusion 9 and below. Fusion 16 only comes in the Studio variant, otherwise you need to use DaVinci Resolve 16 to access the embedded Fusion for free. > Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical…

The most trusted tool since Apple shitcanned Shake, that is.

It's too bad that Shake couldn't be open-sourced. Several big post facilities (ILM, Weta, and others) bought the source code and have probably done great things with it.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#19

Post by BubRoss »

PunksATawnyFill wrote: post #17
mroche wrote: post #14

Earlier quoted context omitted.

> Digital Fusion (much more awkward, but free) Small nitpick, only applies to Fusion 9 and below. Fusion 16 only comes in the Studio variant, otherwise you need to use DaVinci Resolve 16 to access the embedded Fusion for free. > Figuring out why programs like these work well and what gaps are between them and general programs is interesting because many of the reasons come down to pragmatic workflow and not technical…

The most trusted tool since Apple shitcanned Shake, that is. It's too bad that Shake couldn't be open-sourced. Several big post facilities (ILM, Weta, and others) bought the source code and have probably done great things with it.

No one really did much with it. Nuke is actually much better than shake by a wide margin, using shake feels like going back to the stone ages due to interactivity.

Re: Node-based a/v composition: programs as graphs and graphs as compositional tools

#20

Post by kousun12 »

bilekas wrote: post #18

This eternal project is crazy crazy crazy good.. Achieved in the space of only a year.. This is incredible. Beautiful. Really.

thanks! appreciate it. would loved to have spent some more time on it / made it more of a primary thing for me, but i suspect it's a sort of all or none kind of work && i hope someone can get a team together and do a version of it for real
Post reply on HN