Biff.graph: structure your Clojure codebase as a queryable graph

Amazing project, getting into Clojure and will no doubt have use for this as a solo dev

Thanks for mentioning datajet, I'll be taking a look at that for sure...

> From what I hear, the main draw is separating what you want from how you get it, so your calling code can just focus on what it needs. But you can use regular functions to do that. What libraries like Pathom do is leave it open to the caller what shape of data they need.

hmmm... it would be interesting to try an approach where you make heavy use of memoization and then write your functions to take the the minimal set of inputs (e.g. just the primary key for a record). I'm not sure if that's exactly what you had in mind, but here's a strawman example:

  ;; instead of having a resolver with this input
  {:input [:person/age
           :person/name
           {:person/pet [:pet/species
                         :pet/n-legs]}]}
  
  ;; you could have this plain function which calls regular functions to get its
  ;; input, each of which only need a single entity ID for their input
  (defn get-person-stuff [db person-id]
    (let [age         (get-person-age db person-id)
          name        (get-person-name db person-id)
          pet-id      (get-person-pet db person-id)
          pet-species (get-pet-species db pet-id)
          pet-n-legs  (get-pet-n-legs db pet-id)]
      ...))

And you know, I think that would be workable, even though it feels more boilerplatey to me. It would still get you the main benefit of not having to keep track of all the data shapes that are needed by the functions you're calling etc. Some off-the-cuff thoughts:

- with this approach you have a single function for each attribute, so you don't have the situation with pathom/biff.graph where there are multiple resolvers that could be called to get a particular attribute. However note that you could always put an assertion in your codebase that ensures no two resolvers share the same output key, which would then also give you the ability to know exactly what resolvers are being called.

- my example above doesn't include optional inputs, so that's logic you'd also need to write into all your functions: don't fetch the pet data if the pet ID is nil, don't return anything if the person name is nil, etc.

- if you do all that with regular code instead of dependency injection, that does mean you have more code to test, and you have to either supply a test DB (and populate it with everything the functions you're calling need) or mock out the functions. With the dependency injection approach you get plain-old-pure-functions which helps keep your unit tests nice and dumb.

- I like the readability of being able to look at the input / output queries and know exactly what shape of data I'm dealing with.

- There might be performance issues with the memoized functions approach. Pathom and biff.graph both support batch resolvers for example, and I'm not sure if you could do the equivalent as cleanly with the functions approach. And Pathom of course has its additional query planning step which does... stuff.

Going back to your comment, some thoughts:

> But I think letting the caller do subtle query changes that can completely change which resolvers are triggered and how something is fetched is kinda leaky.

This is an area where you might like biff.graph more than Pathom. Since there's no query planning step, the way that biff.graph executes your queries should be fairly predictable. It's basically just doing a depth-first traversal of your query.

(My first bullet point above is relevant too--you can always restrict yourself to having only one resolver per attribute so there's no question of what resolver is getting used.)

> How do you write the perfect resolver for all situations? How do you keep them from accidentally exploding their fetches?

Typically you write resolvers with only one level of joins/nesting and then let the query engine do the rest. so e.g. instead of writing a resolver that returns something like `{:person/pet {:pet/id 1, :pet/toys [{:toy/id 2, ...}, ...]}}`, you would have one resolver that returns `{:person/pet {:pet/id 1}}` and then another resolver that takes a pet ID and returns `{:pet/toys [{:toy/id 2}, ...]}` etc.

So there is a trade-off here in that e.g. you may end up running multiple database queries even though you could've stuffed everything you need into a single database query. That is mitigated by batch resolvers at least so you don't get N+1 query problems.

I've never needed to do this myself yet, but if you do run into any places where the performance isn't good enough, you can always write those bits the regular way (e.g. have a resolver that does a more complex query and returns nested data and/or don't even use pathom/biff.graph for this one bit). i.e. optimize where needed but stick with the default in most places.

> Is it not better to have things be explicit through function calls instead of chasing down disjointed call graphs?

There are pros and cons I think. Sometimes you want to know how an input is being computed and sometimes you want to be able to understand some logic in isolation. In practice I've acclimated quite a bit to the graph structure; I feel like it does a nice job of helping you split your code into the right "chunks".

Oh sorry, I kind of meant it the other way around. You start with biff.graph and you'd swap in Pathom if the featureset or performance wasn't adequate. It makes sense that since you support a subset that it doesn't work the other way around! It might make sense to reinvent the wheel if there is a clear gain - but if there isn't any big innovation going on in the library interface, it's generally nice to keep the same interface if it's easy enough to do - but that's just my opinion haha

And yeah, now that I have a larger application with Pathom.. I should retry Viz too :))

And that's very cool you're taking error seriously. Sorry, I missed it when I looked at the rep the first time! Thanks for your help with Pathom a few months back (kxygk on Github)

ah got it. yeah, in that case you can write a biff -> pathom resolver shim that works for everything. Again though the main thing is just the fact that they have two completely different query engines and aren't guaranteed to give the same results. e.g. off the top of my head I can think of a contrived scenario where biff.graph might not be able to resolve something but Pathom can since it can "look ahead" in the query planning step.

Maybe that kind of situation is fine and the question is really just if there are queries that biff.graph can handle which Pathom can't. If your resolvers are written correctly maybe not? But there have definitely been times with Pathom where I did something wrong that threw off the query planner in ways I didn't expect.

In any case, if I end up wanting to support migrating easily between the two as a core feature, I'd definitely want to e.g. do a bunch of generative tests to find out what kinds of queries end up with different results. Until then, a downside of supporting Pathom resolvers without a shim is that it might give people the false impression that biff.graph is a drop-in replacement for Pathom or vice-versa.

So far though the main target audience I have for biff.graph is people (biff users) who have never even heard of Pathom before, so interchangeability hasn't been a top concern. Though if many people start using biff 2 and then eventually some of the start wanting to migrate to pathom, I'd be down to explore that area.

And haha yeah nice to bump into you again--I think I remembered your username from reddit, assuming it's geokon there.