Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flow-based Programming

jpaulmorrison.com

Post reply on HN
Print view View original

21–30 of 34 posts

Re: Flow-based Programming

#21

Post by amelius »

seanmcdirmid wrote: post #20
amelius wrote: post #11

Earlier quoted context omitted.

I think the biggest problems with flow based or reactive programming are the occurrence of glitches, and, more importantly, the problem of how to compute updates without performing unnecessary work. I wonder if these problems have been successfully addressed somewhere.

My work [1] addresses glitches head on (hence the system is called "Glitch") by tolerating glitches and at least making the extra work functionally benign (via logging and rollback, like a transaction). Glitch is not based on data flow, however, but somewhere in between even if still very reactive. I've built a few data flow systems in the past (my dissertation work [2] was based on one), but I decided to tolerate ra…

Interesting. Do you know of any work that computes a data flow graph as the program runs (i.e. data flow is implicit in the source code), which performs then reactively?

Re: Flow-based Programming

#22

Post by seanmcdirmid »

peterlk wrote: post #2

What specifically differentiates this from functional reactive programming?

Not really related. FRP has as much in common with dataflow programming as generic functional programming does. Related incidentally in that both refuse side effects, but one is based on function application while the other is based on data flow connections.

I'm not really familiar with how flow-based programming differs from data flow programming, however.

Re: Flow-based Programming

#23

Post by seanmcdirmid »

amelius wrote: post #21
seanmcdirmid wrote: post #20

Earlier quoted context omitted.

My work [1] addresses glitches head on (hence the system is called "Glitch") by tolerating glitches and at least making the extra work functionally benign (via logging and rollback, like a transaction). Glitch is not based on data flow, however, but somewhere in between even if still very reactive. I've built a few data flow systems in the past (my dissertation work [2] was based on one), but I decided to tolerate ra…

Interesting. Do you know of any work that computes a data flow graph as the program runs (i.e. data flow is implicit in the source code), which performs then reactively?

Ya, that is exactly how Glitch works, except I just call the data flow graph as a dependency graph :).

Computing the dependency graph dynamically obviously leads to a lot more flexibility. I'm not really sure what you mean by "reactively" here: if you mean does it reactively update computations as state (and even code, if you want to get Bret Victor about it) changes, then ya.

Data flow by itself isn't necessarily reactive; actually the way it was originally defined to work very lazy push things through. Continuous interpretations where changes were propagated non-lazily didn't come until much later.

Re: Flow-based Programming

#29

Post by SEJeff »

harlowja wrote: post #14

A framework that is similar to this in python (that I am one of the authors of) [1] (and [2] for more in-depth docs/examples). It's used in openstack (which some people may have heard of) where appropriate (and where desirable...) to help in consistent, scalable execution using a flow like methodology to declaratively describe workflows (and execute them using various strategies/engines). [1] https://github.com/opens…

Just gonna say that task flow is wonderful software that is usable outside of the giant openstack ecosystem. I use it for some internal projects and am quite pleased with it.

Re: Flow-based Programming

#30

Post by i000 »

samuell wrote: post #16
harlowja wrote: post #14

A framework that is similar to this in python (that I am one of the authors of) [1] (and [2] for more in-depth docs/examples). It's used in openstack (which some people may have heard of) where appropriate (and where desirable...) to help in consistent, scalable execution using a flow like methodology to declaratively describe workflows (and execute them using various strategies/engines). [1] https://github.com/opens…

The one I did know in python was PaPy: https://code.google.com/p/papy/ Will have to have a closer look at this one as well, though.

Author of PaPy here.

Thanks for giving PaPy a try. Although PaPy [https://github.com/mcieslik-mctp/papy] has never gained traction :(, I have been using it daily for over 5 years (no bugs in the scheduler so far). By now our PaPy based genomics pipelines have probably processed petabytes of data.

Post reply on HN