Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The plan-execute pattern

mmapped.blog

Post reply on HN
Print view View original

31–40 of 81 posts

Re: The plan-execute pattern

#31

Post by onetimeuse92304 »

Used something very similar many times in my past without knowing it is formalised as a pattern.

For example, one application of this was a long migration project where a large collection of files (some petabytes of data) was to be migrated from an on-prem NAS to cloud filesystem. The files on NAS were managed with additional asset management solution which stored metadata in a PostgreSQL (actual filenames, etc.)

The application I wrote was composed of a series of small tools. One tool would contact all of the sources of information and create a file with a series of commands (copy file from location A to location B, create a folder, set metadata on a file, etc.)

Other tools could take that large file and run operations on it. Split it into smaller pieces, prioritise specific folders, filter out modified files by date, calculate fingerprints from actual file data for deduplication, etc. These tools would just operate on this common file format without actually doing any operations on files.

And finally tool that could be instantiated somewhere and execute a plan.

I designed it all this way as it was much more resilient process. I could have many different processes running at the same times and I had a common format (a file with a collection of directives) as a way to communicate between all these different tools.

Re: The plan-execute pattern

#32

Post by munificent »

bradleybuda wrote: post #24

I use this pattern a lot (and I often find myself wishing I used it more). Another way to think about this is "noun-ification" (aka the Command pattern) - instead of invoking a function directly, capture all of its arguments, serialize them to some "store", then provide the ability to rehydrate them and actually run it.

I came here to say basically the same thing. To me, this reads like an example of the Command pattern, which is one of my favorite, most-used patterns.

The idea of taking some chunk of imperative behavior and reifying it into an object that can perform that behavior is very powerful:

* It allows you to attach other metadata to the object so that code can ask questions about it: "Do you perform IO?" "Can you be reverted?" It lets surrounding code introspect over operations in useful ways.

* It allows you to separate the code that creates the command from the code that performs it (what this blog post is about). This can nicely decouple "strategy"-type code from "execution harness" type code. Both of those are often fairly complex in their own right, so a pattern that lets you build a nice abstraction layer between them can be very helpful.

* It allows the command object to support multiple related operations. The most common example, which I deeply love, is implementing undo. A Command object for some UI operation can also support a separate method to undo the same operation. Your undo stack is then just a list of these Command objects.

Re: The plan-execute pattern

#33

Post by touisteur »

PaulHoule wrote: post #10
Spivak wrote: post #5

Earlier quoted context omitted.

> Scheme isn’t special because it has closures, it is special because you can write functions that write functions. I swear this informational parasite will never die, any language with eval can write functions that write functions. What makes Scheme and other homoiconic languages special is that the code that is currently executing can be manipulated live like any other data structure and it works.

Back in the 1980s I found a book at the public library where somebody had a BASIC program that would read a BASIC program without line numbers and with structured programming loops and rewrite it an ordinary BASIC. It was oriented towards CP/M so I typed the program in and modified it to run on my TRS-80 Coco. It was kinda like https://en.wikipedia.org/wiki/Ratfor Similarly I've written awk programs that write shell…

Very common for me to write shell scripts that generate shell scripts to run.

Instead of creating gnuplot scripts to loaded, sometimes a good sequence of echo "set terminal png 800,600; plot..." | gnuplot > my.png

or a very long imagemagick drawing command or a simple svg generated then converted to png then assembled (thank montage), can do the job.

The whole advantage of generating intermediate shell scripts, is to replay one part specifically to debug a small part of a script pipeline.

Re: The plan-execute pattern

#34

Post by munificent »

scotty79 wrote: post #6
advael wrote: post #4

I spent the first bit of this article going "Wait isn't that just what a finite state machine is?" and then was subsequently super satisfied by the next bit where they say that this is indeed a really good way to handle the general case Reducing program logic to a small state machine and business logic that interacts with it in predefined ways is a pattern I learned while doing asynchronous netcode for a game project…

It's quite sad that generał purpose language designers are not yet at the stage of daring to provide any convenient syntax for high level concepts like state machines or entity component systems.

UnrealScript has (had?) built-in syntactic support for state machines.

I think Jonathan Blow had some ideas for language support for ECS in Jai, or at least support for structure-of-array stuff. I'm not sure if it panned out.

My experience with both of those patterns is that each implementation tends to have enough unique requirements that it's hard for a language to find a sweet spot where built-in syntax can cover enough real-world use cases.

For example, off the top of my head, with state machines I've had questions like:

* Can states be defined dynamically or are they all known at compile time?

* Likewise, is the set of possible transitions fixed (and declarative) or freely specified at runtime?

* Does every instance of a given state machine always have the same set of states or do some vary?

* Can you attach arbitrary data to states? Arbitrary behavior?

* Can you attach arbitrary data to state transitions? Arbitrary behavior?

* Is the predicate logic for determining when to perform a state transition declarative and built-in, or can arbitrary imperative code cause a state transition?

* Are transitions synchronous? Asynchronous? Either?

* Can outside code observe when transitions occur in some way?

* Is the current state publicly accessible or encapsulated?

* Are states first-class objects? Are transitions?

I've tried just to build a reusable FSM library and even that wasn't very successful because there's just so much variance in each implementation.

Re: The plan-execute pattern

#35

Post by codeflo »

rmckayfleming wrote: post #13

"I feel uneasy about design patterns. On the one hand, my university class on design patterns revived my interest in programming. On the other hand, I find most patterns in the Gang of Four book to be irrelevant to my daily work; they solve problems that a choice of programming language or paradigm creates." My relationship with design patterns changed when I stopped viewing them as prescriptive and started viewing t…

Descriptive of the kinds of workarounds that a lack of proper abstraction capabilities made necessary in ancient Java.

Re: The plan-execute pattern

#37

Post by scotty79 »

munificent wrote: post #34
scotty79 wrote: post #6

Earlier quoted context omitted.

It's quite sad that generał purpose language designers are not yet at the stage of daring to provide any convenient syntax for high level concepts like state machines or entity component systems.

UnrealScript has (had?) built-in syntactic support for state machines. I think Jonathan Blow had some ideas for language support for ECS in Jai, or at least support for structure-of-array stuff. I'm not sure if it panned out. My experience with both of those patterns is that each implementation tends to have enough unique requirements that it's hard for a language to find a sweet spot where built-in syntax can cover…

> Can states be defined dynamically or are they all known at compile time?

You can say that about classes as well, give different answer and have languages that support one of those or both.

Similar questions exist about a lot of concepts that landed in some or many languages, like objects, async, events, promises, continuations. In most of them we achieved some sort of fuzzy consensus of what their capabilities should be.

Re: The plan-execute pattern

#38

Post by scotty79 »

wavemode wrote: post #20
scotty79 wrote: post #6

Earlier quoted context omitted.

It's quite sad that generał purpose language designers are not yet at the stage of daring to provide any convenient syntax for high level concepts like state machines or entity component systems.

I would argue that the domain of language design is quite a bit lower-level than the domain of specific design patterns like state machines or ECS. A language that wants to support ECS, shouldn't implement ECS as part of its syntax. It should provide features that allow library developers to implement ECS (in an ergonomic and efficient way).

Why not? When defining a class I should have to be able to have implicit collection of objects of this class with each field of the class kept in its own separate array. It's about memory organisation. And languages certainly feel free to impose those with stack, heap and colocation of all fields of an instance regardless of how varied they are in meaning and usage.

Re: The plan-execute pattern

#39

Post by ijustlovemath »

I do this for a code generation tool we use internally! Plans contain all the metadata to actually do a thing, but allow transformation, filtering etc before execution. It also forces you to abstract your code in a cleaner way, and minimize the surface of your plan generation/execution functions, which makes them infinitely more testable.

The way I think about when to use a plan is whenever you're doing batches of I/O of some kind, or anything that you might want to make idempotent.

Re: The plan-execute pattern

#40

Post by stevesimmons »

caseyohara wrote: post #27
bradleybuda wrote: post #24

I use this pattern a lot (and I often find myself wishing I used it more). Another way to think about this is "noun-ification" (aka the Command pattern) - instead of invoking a function directly, capture all of its arguments, serialize them to some "store", then provide the ability to rehydrate them and actually run it.

Interesting you used the term hydrate because I only hear that used when talking about front-end web development (specifically Next and React) and it's a relatively new term. The "Hydration (web development)" Wikipedia article was only created in December 2020. https://en.wikipedia.org/wiki/Hydration_(web_development) I feel like serialize/deserialize or marshal/demarshal are more common in broader computer science c…

"Hydrate" is much older than 2020. Here's a StackOverflow reference from 2011 [1], for instance. I first heard it used around that time, for loading objects from databases (in a banking context).

[1] https://stackoverflow.com/a/4929478

Post reply on HN