Live data from Hacker News

Simplify your code: Functional core, imperative shell

testing.googleblog.com

141–150 of 219 posts

Re: Simplify your code: Functional core, imperative shell

#141

I wrote our AI agents code with a functional core + imperative shell and I have to agree: this approach yields much faster cycle times because you can run pure unit tests and it makes testing a lot easier. We have tens of thousands of lines of code for the platform and millions of workflow runs through them with no production errors coming from the core agent runtime which manages workflow state, variables, rehydrati…

Great examples. We were taught to pass variables, scalar or compound, into API's. Most of us were never taught to pass functions.

Even Python examples in trainings that look functional might not be. They put the function calls in as arguments. The beginner thinks the function returns some data, that would be in a variable, and they are implicitly passing that variable. Might as well, for readability, do the function call first to pass a well-named variable instead.

That was my experience. That plus minimizing side effects in functions. I've yet to really learn functional programming where I'd think to pass a function in an API. What are the best articles or books for us to learn that in general or in Python?

Re: Simplify your code: Functional core, imperative shell

#142
post #114
post #112

Earlier quoted context omitted.

> I think they are orthogonal at worst, and in agreement at best. I have considered them being orthogonal, but then the definition of the "shell" and "core" becomes problematic in this comparison. What you call shell in GCSS is not shell in FCIS at all, more like a boundary. Even there it is questionable whether boundary should be more specific than the core. At the core, things can be more integrated than at the bou…

> then the definition of the "shell" and "core" becomes problematic in this comparison I agree -- if you're trying to make the words "shell" and "core" mean the same things between FCIS and GCSS, or identify the same parts of a program, then there will be problems. I think FCIS and GCSS are just two different ways of analyzing a program into pieces. Just as the number 8 can be seen through addition as 3 + 8 and throu…

"Words are allowed to have contextual meanings"

Sure, but this discussion is about FCIS, that's the context, and the GP should consider that.

" think I need a more precise meaning of "business logic" before I can answer this question"

Well, some examples. A tax application - the tax calculation according to the law. A word processor - layouting and rendering engine. A video game - something that calculates the state of the world, according to the rules of the game.

So a game is a good example where the core can be more specialized than the shell. You can imagine a generic UI library shared by a bunch of games, but a generic game rules engine - that's just a programming language.

"Decomposition is a fundamental part of software engineering: we decompose a large problem into smaller ones, solve those, them compose those solutions into a solution to the large problem"

There is a big misconception in SW engineering that the above decomposition always exists in a meaningful way. Take the tax calculation for example. That cannot be decomposed into pieces that are generic, and potentially reusable elsewhere. It's just a list of rules and exceptions that need to be implemented as stated. You can decompose it into "1st part of calculation" and "2nd part of calculation", but that's meaningless (unhelpful). (Similarly for the game example above, the rules only exist in the context of other rules.)

Surprisingly many problems are like that, and that makes them kinda difficult to test.

Re: Simplify your code: Functional core, imperative shell

#143

I never liked encountering code that chains functions calls together like this email.bulkSend(generateExpiryEmails(getExpiredUsers(db.getUsers(), Date.now()))); Many times, it has confused my co-workers when an error creeps in in regards to where is the error happening and why? Of course, this could just be because I have always worked with low effort co-workers, hard to say. I have to wonder if programming should ha…

Ada is a great modern language that preserves the distinction between functions and procedures that you mention.

Re: Simplify your code: Functional core, imperative shell

#144
post #112

Earlier quoted context omitted.

> I think they are orthogonal at worst, and in agreement at best. I have considered them being orthogonal, but then the definition of the "shell" and "core" becomes problematic in this comparison. What you call shell in GCSS is not shell in FCIS at all, more like a boundary. Even there it is questionable whether boundary should be more specific than the core. At the core, things can be more integrated than at the bou…

One way to get some intuition with FCIS is to write some Haskell. Because Haskell programs pretty much have to be FCIS or they won't compile. How it plays out is... 1. A Haskell program executes side effects (known as `IO` in Haskell). The type of the `main` program is `IO ()`, meaning it does some IO and doesn't return a value - a program is not a function 2. A Haskell program (code with type `IO`) can call function…

I agree with the suggestion to study Haskell, I like Haskell quite a lot (although I don't write applications in it).

Re: Simplify your code: Functional core, imperative shell

#145
post #89

Earlier quoted context omitted.

Not a single person in this thread commented on the use of Date.now() and similar - surely clock.now() - you never ever want to use global time in any code, how could you test it? clock in this case is a thing that was supplied to the class or function. It could just be a function: () -> Instant. (Setting a global mock clock is too evil, so don't suggest that!)

I was just referring to how pipes make these kinds of chained function calls more readable. But on your point, I think using Date.now() is perfectly ok.

> I think using Date.now() is perfectly ok.

This is why we have tests which we need to update every 3 months, because somebody said this. This is of course, after a ton of research went into finding out why the heck our tests broke suddenly.

Re: Simplify your code: Functional core, imperative shell

#146
post #71
post #65

Earlier quoted context omitted.

I have not seen too many (any?) times where the monad trick is done in such a way that they don't combine everything in a single context wrapper and talk about the "abnormal" case where things don't complete during execution. Granted, in trying to find some examples that stick in my memory, I can't really find any complete examples anymore. Mayhap I'm imagining a bad one? (Very possible.)

You would deal with this problem in the same way you would with, say, in a REST API. If the transaction object is serializable you can just store it in a DB, for example. If it's some C++ pointer from some 3rd-party library that you can't really serialize and gotta keep open, you gotta keep it in memory and manage its lifetime explicitly, be it a REST web server, in Haskell or in a C++ app.

Right, I think a better way of stating my main assertion here is that you have to be able to partially work in a transaction. If your "shell" pretends that you can always complete the full transaction, either successfully or with a failure, then it is a brittle shell. Sometimes, you can simply make progress on an presumed open transaction.

Re: Simplify your code: Functional core, imperative shell

#147
post #123

Earlier quoted context omitted.

Surely transactions are a pretty good example of where functional core / imperative shell is a good guide. You really don't want to be doing arbitrary side effects inside your transaction because those can't be backed out. Check out STM in Haskell for a good example.

> You really don't want And that's what this thread is filled with, and that's what I'm pushing back against. > the RAI pattern is nice > indicate if you are in different sections is nice Style doesn't matter, flavour doesn't matter, wants don't matter, code "quality" (whatever that means) doesn't matter, niceness doesn't matter. A transaction can be rolled back. If it can't, it's not a transaction.

I'd go a little further, though? Transactions have to be able to fail at the very last step. That could just be the "commit" stage. Everything up to that point could have been perfectly fine, but at commit it fails. More, the time of execution of that commit could be fairly far removed from the time of starting the transaction.

To that end, any style that tries to move those two time periods closer together in code is almost doomed to have some either hard to reason about code, or some tough edge cases that are hard to specify.

(Granted, I'll note that most transactions that people are dealing with on a regular basis probably do open and close rather close to each other.)

Re: Simplify your code: Functional core, imperative shell

#148
post #136

Earlier quoted context omitted.

With a good library you could do just that, by having the functions return only queries and then expand them to the actual values (by interacting with the DB) after applying the filtering to it?

So would you then have to do `getActualUsers(db.getUsers())` or `query(db.getUsers())`? Still smells like in such a case the developer avoids the complications of abstraction or OOP by making the user deal with it. That's bad API design due to putting ideology before practicality or ergonomics.

You would have an API that makes the query shape, the query instance with specific values, and the execution of the query three different things. My examples here are SQLAlchemy in Python, but LINQ in C# and a bunch of others use the same idea.

The query shape would be:

  active_users = Query(User).filter(active=True)
That gives you an expression object which only encodes an intent. Then you have the option to make basic templates you can build from:

  def active_users_except(exclude):
      return active_users.filter(User.id.not_in(exclude)
...where `exclude` is any set-valued expression.

Then at execution time, the objects representing query expressions are rendered into queries and sent to the database:

  exclude_criterion = rude_users()  # A subquery expression
  polite_active_users = load_records(
      active_users_except(exclude_criterion)
  )
With SQLAlchemy, I'll usually make simple dataclasses for the query shapes because "get_something" or "select_something" names are confusing when they're not really for actions.

  @dataclass
  class ActiveUsers(QueryTemplate):
      active_if: Expression = User.active == true()

      @classmethod
      excluding(cls, bad_set):
          return cls(
              and_(
                  User.active == true(),
                  User.id.not_in(bad_set)
              )
          )

      @property
      def query(self):
          return Query(User).filter(self.active_if)

  load_records(
      ActiveUsers.excluding(select_alice | select_bob).query
  )

Re: Simplify your code: Functional core, imperative shell

#149
post #148

Earlier quoted context omitted.

So would you then have to do `getActualUsers(db.getUsers())` or `query(db.getUsers())`? Still smells like in such a case the developer avoids the complications of abstraction or OOP by making the user deal with it. That's bad API design due to putting ideology before practicality or ergonomics.

You would have an API that makes the query shape, the query instance with specific values, and the execution of the query three different things. My examples here are SQLAlchemy in Python, but LINQ in C# and a bunch of others use the same idea. The query shape would be: active_users = Query(User).filter(active=True) That gives you an expression object which only encodes an intent. Then you have the option to make bas…

As in Django querysets. But starts to get messy with complex queries.

Re: Simplify your code: Functional core, imperative shell

#150
post #142
post #114

Earlier quoted context omitted.

> then the definition of the "shell" and "core" becomes problematic in this comparison I agree -- if you're trying to make the words "shell" and "core" mean the same things between FCIS and GCSS, or identify the same parts of a program, then there will be problems. I think FCIS and GCSS are just two different ways of analyzing a program into pieces. Just as the number 8 can be seen through addition as 3 + 8 and throu…

"Words are allowed to have contextual meanings" Sure, but this discussion is about FCIS, that's the context, and the GP should consider that. " think I need a more precise meaning of "business logic" before I can answer this question" Well, some examples. A tax application - the tax calculation according to the law. A word processor - layouting and rendering engine. A video game - something that calculates the state…

> Take the tax calculation for example. That cannot be decomposed into pieces that are generic, and potentially reusable elsewhere.

Quite right! However, the tax code does change with some regularity, and we can expect that companies like Intuit should have gotten quite good by now -- even on a pure profit motive -- at making it possible to relatively quickly modify only the parts of their products that require updating to the latest tax code. To put it another way, while it might be the case that the tax code for any given year is not amenable to decomposition, all tax codes within a certain span of years might be specific instances of a more general problem. (I recall a POPL keynote some years back that argued for formalizing tax codes in terms of default logic!) By solving that general problem, you can instantiate your solution on the given year's tax code without needing to recreate the entire program from scratch.

To be clear, I'm the one who brought subproblem decomposition into the mix, and we shouldn't tar the top-level commenter with that brush unnecessarily. Of course some problems will be un-decomposable "leaves". I believe their original point, about a specific business layer sitting on top of a more general core, still applies.

> So a game is a good example where the core can be more specialized than the shell. You can imagine a generic UI library shared by a bunch of games, but a generic game rules engine - that's just a programming language.

As it happens, the "ECS pattern" (Entity, Component, and System) is often considered to be a pretty good way of conceptualizing the rules of a game. An ECS framework solves the general problem (of associating components to entities and executing the systems that act over them), and a game developer adapts an ECS framework to their specific needs. The value in this arrangement is precisely that, as the game evolves and takes shape, only the logic specific to the game needs to be changed. The underlying ECS framework is on the whole just as appropriate for one game as for any other.

(I could also make a broader point about game engines like Unity and Unreal, and how so many games these days take the "general" problem solved by these engines and adapt them to the "specific" problem of their particular game. In general, nobody particularly wants to make engine-level changes for each experiment during the development of a game, even though sometimes a particular concept for a game demands a new engine.)

> Sure, but this discussion is about FCIS, that's the context, and the GP should consider that.

I understood the original commenter as criticizing FCIS (or at least the original post, as "grasping for straws") and suggesting that GCSS is generally more appropriate. In that context, I think it's natural to interpret their use of "core" and "shell" as competing rather than concordant with FCIS.

Post reply on HN