Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kraken Technologies: How we organise our large Python monolith

blog.europython.eu

Post reply on HN
Print view View original

1–10 of 41 posts

Re: Kraken Technologies: How we organise our large Python monolith

#3

Post by bitdivision »

photonthug wrote: post #2

Never heard of https://import-linter.readthedocs.io/ before. Not sure if I like this type of solution, but it's interesting, and certainly the problem is real.

The author, David Seddon, is also the author of the package [0]. So I'm guessing Kraken was an initial user.

0: https://import-linter.readthedocs.io/en/stable/authors.html

Re: Kraken Technologies: How we organise our large Python monolith

#4

Post by CraigJPerry »

This is java circa 2009 flavoured, this model was a popular approach in the java community around that timeframe. Was enforcer the name of the maven plugin? I can’t remember now, but it became fairly popular.

I wonder if this might be susceptible to the same issues that we had in java, specifically it doesn’t prevent accidental duplication, it makes refactoring harder/slower and it’s problematic for the team when it unexpectedly doesn’t catch a misuse - and this happens because the rules enforcement can only happen effectively if the rules themselves are perfect and complete and that gets harder to guarantee with scale and speed of iteration.

For my money today, there’s another model “polylith” which is similar in some ways and as easy to understand but maybe simpler to evolve/change your mind over time.

Re: Kraken Technologies: How we organise our large Python monolith

#5

Post by photonthug »

CraigJPerry wrote: post #4

This is java circa 2009 flavoured, this model was a popular approach in the java community around that timeframe. Was enforcer the name of the maven plugin? I can’t remember now, but it became fairly popular. I wonder if this might be susceptible to the same issues that we had in java, specifically it doesn’t prevent accidental duplication, it makes refactoring harder/slower and it’s problematic for the team when it…

> this happens because the rules enforcement can only happen effectively if the rules themselves are perfect and complete and that gets harder to guarantee with scale and speed of iteration.

IMHO, project-level assertions about initialization-order feels a little bit dirty, but potentially practical as long as there's only a few rules. Surely even a bazillion line monolith can still often be thought of in terms of a few layers. But by the time you get to lots of rules it's getting hard to stomach the code-smell and keep justifying it as practical. At that point I'd want something dynamic and backed by a real solver.. not just a layer of static-analysis.

The deal framework for DBC doesn't quite do this afaik but is starting to expose some stuff along these lines, and at least can assert imports are "pure". https://deal.readthedocs.io/details/module_load.html

Re: Kraken Technologies: How we organise our large Python monolith

#7

Post by bnert »

CraigJPerry wrote: post #4

This is java circa 2009 flavoured, this model was a popular approach in the java community around that timeframe. Was enforcer the name of the maven plugin? I can’t remember now, but it became fairly popular. I wonder if this might be susceptible to the same issues that we had in java, specifically it doesn’t prevent accidental duplication, it makes refactoring harder/slower and it’s problematic for the team when it…

polylith ftw.

Been learning some of the idioms of a polylith architecture using the `poly` tool + clojure. Really enjoying the idioms/practices it encourages.

Re: Kraken Technologies: How we organise our large Python monolith

#8

Post by loveparade »

mint2 wrote: post #6

Note, this is not the crypto kraken, it’s a different kraken for those who ignore crypto related stuff

Why does it matter? Just because I don't like Google doesn't mean they suck at writing Python code. The same goes for anything crypto.

Re: Kraken Technologies: How we organise our large Python monolith

#10

Post by gyulai »

I quite like layered dependency structures.

I even have a lua codebase where I've linearized the sourcecode files into a particular order. The layout looks like this:

  01_foo/  
    03_foo.lua  
    04_bar.lua  
    15_whatever.lua  
  02_bar/  
    22_whatever.lua  
    56_something.lua
It has always bothered me in codebases that they don't have a "beginning" and an "end", and therefore it's difficult for a reader to know where to start when they just want to read the codebase.

So, my approach is to impose this linear order and only allow code that comes later to depend on code that comes earlier, never the other way around. This way, you can read through the code rather easily. As you look at any piece of code, it will only depend on code you've already looked at, so you don't have to be constantly jumping through the codebase as you're reading.

I've also found that failure to correspond to a linearizable dependency structure is a "code smell", and that, as I try to eradicate that code smell, I frequently end up with code that's better in all kinds of ways.

Post reply on HN