Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Architecture Patterns with Python

cosmicpython.com

Post reply on HN
Print view View original

131–140 of 143 posts

Re: Architecture Patterns with Python

#131

Post by deergomoo »

kelafoja wrote: post #16
BerislavLopac wrote: post #15

Some parts of this book are extremely useful, especially when it's talking about concepts that are more general than Python or any other specific language -- such as event-driven architecture, commands, CQRS etc. That being said, I have a number issues with other parts of it, and I have seen how dangerous it can be when inexperienced developers take it as a gospel and try to implement everything at once (which is a c…

Could you explain how repository pattern is a "huge overkill that adds complexity with very little benefit"? I find it a very light-weight pattern and would recommend to always use it when database access is needed, to clearly separate concerns. In the end, it's just making sure that all database access for a specific entity all goes through one point (the repository for that entity). Inside the repository, you can d…

In my experience, both SQL and real-world database schema are each complex enough beasts that to ensure everything is fetched reasonably optimally, you either need tons of entity-specific (i.e. not easily interface-able) methods for every little use case, or you need to expose some sort of builder, at which point why not just use the query builder you're almost certainly already calling underneath?

Repository patterns are fine for CRUD but don't really stretch to those endpoints where you really need the query with the two CTEs and the four joins onto a query selecting from another query based on the output of a window function.

Re: Architecture Patterns with Python

#132

Post by dkarl »

globular-toast wrote: post #113
dkarl wrote: post #79

Earlier quoted context omitted.

> Turns out all the big ones with strict architectural (n=3) pattern usage, although “clean”, the code is waaaay to complex and unnecessarily slow in tasks that at first glance should had been simple. My last job had a Python codebase just like this. Lots of patterns, implemented by people who wanted to do things "right," and it was a big slow mess. You can't get away with nearly as much in Python (pre-JIT, anyway) a…

> What bothers me about this book and other books that are prescriptive about application architecture is that it pushes people towards baking in all the complexity right at the start, regardless of requirements, instead of adding complexity in response to real demands. The trouble is if you strictly wait until it's time then basically everything requires some level of refactoring before you can implement it. The dre…

> The dream is that new features is just new code, rather than refactoring and modifying existing code

I don't just mean new features. I mean new cross-cutting capabilities. I mean emitting metrics from an application that has never emitted metrics. I also mean adding new dimensions to existing capabilities, like adding support for a second storage backend to an application that has only ever supported one database.

These are changes that I was always taught were important to anticipate. If you don't plan ahead, it'll be near impossible to add later, right? After a couple of decades of working on real-life codebases, seeing the work that people pour into anticipating future needs, making things pluggable, all that stuff, seeing exactly how helpful that kind of up-front speculative work turns out to be in practice when a real need arises, and comparing it to the work required to add something to a codebase that was never prepared for it, I have become a staunch advocate for skipping almost all of it.

> Unfortunately in my experience people follow by example and the frog can boil for a long time before people start to realise that their time is spent mostly doing large refactors because the code just doesn't support the kind of flexibility and extensibility they need

If the engineers are doing large refactors, what in the world could they be doing besides adding the "kind of flexibility and extensibility they need?"

One thing to keep in mind when you compare two options is that unless the options involve different hiring strategies, the people executing them will be the same. If you have developers doing repeated large refactors without being able to make the codebase serve the current needs staring them in the face, what do you think will happen if you ask them to prepare a codebase for uncertain future needs? It's a strictly harder problem, so they will do a worse job, or at least no better.

Re: Architecture Patterns with Python

#133

Post by exe34 »

procaryote wrote: post #118
globular-toast wrote: post #54

Earlier quoted context omitted.

DDD isn't about objects. It's just about modelling the domain (real world) using the tools available to you. Some things are best modelled by objects, some are best modelled by functions or other constructs. The real point is establish a common language to talk about the domain. This is enormously powerful. Have you ever worked with people who don't speak your language? Everything takes 3x as long as ideas aren't com…

Good coders learn enough about the business to check the code. Domain experts can look at the running software. With DDD you just get a third model that is neither the domain, nor the software, and both the domain experts and the programmers will have to work extra to maintain and understand it. Worse, people often try to build this model up front, which means it will be wrong, hard to implement and probably get thro…

I think people are getting triggered by the word domain, and conflating it with a particular cargo cult called DDD. It's the same with agile - the one that's implemented is usually the cargo cult version that charges you the cost of the "official" process without the benefits.

I meant domain modelling in the simplest sense: I created three objects, the market, the trading strategy and a simulated version of the market. that's it. no paperwork, no forms filled in triplicate.

Re: Architecture Patterns with Python

#134

Post by _dain_ »

zahlman wrote: post #128
_dain_ wrote: post #119

Earlier quoted context omitted.

>The point is that I gain no information from it No, you do gain information from it: that the function takes an Iterable[Ducklike]. Moreover, now you can tell this just from the signature , rather than needing to discover it yourself by reading the function body (and maybe the bodies of the functions it calls, and so on ...). Being able to reason about a function without reading its implementation is a straightforwa…

>No, you do gain information from it: that the function takes an Iterable[Ducklike]. I already had that information. I understand my own coding style. >Being able to reason about a function without reading its implementation is a straightforward win. My function bodies are generally only a few lines, but my reasoning here is based on the choice of identifier name. Yes, it takes discipline, but it's the same kind of d…

>I already had that information. I understand my own coding style.

Good for you, but you're not the only person working on the codebase, surely.

>My function bodies are generally only a few lines, but my reasoning here is based on the choice of identifier name.

Your short functions still call other functions which call other functions which call other functions. The type will not always be obvious from looking at the current function body; often all a function does with an argument is forward it along untouched to another function. You often still need to jump through many layers of the call graph to figure out how something actually gets used.

An identifier name can't be as expressive as a type without sacrificing concision, and can't be checked mechanically. Why not be precise, why not offload some mental work onto the computer?

>Yes, it takes discipline, but it's the same kind of discipline as adding type annotations.

No, see, this is an absolutely crucial point of disagreement:

Adding type annotations is not "discipline"!

Or at least, not the same kind of discipline as remembering the types myself and running the type checker in my head. The type checker is good because it relieves me of the necessity of discipline, at least wrt to types.

Discipline consumes scarce mental effort. It doesn't scale as project complexity grows, as organizations grow, and as time passes. I would rather spend my limited mental effort on higher level things; making sure types match is rote clerical work, entirely suitable to a machine.

The language of "discipline" paints any mistake as a personal/moral failure of an individual. It's the language of a blame-culture.

Re: Architecture Patterns with Python

#135

Post by _dain_ »

pansa2 wrote: post #125
_dain_ wrote: post #120

Earlier quoted context omitted.

So don't do this then? The type system does not have to be sound to be useful; Typescript proves this.

> So don't do this then? Don't do what? - Don't write unsound code? There's no way to know until you run the program and find out your `int` is actually a `list`. - Don't assume type annotations are correct? Then what's the point of all the extra code to appease the type checker if it doesn't provide any guarantees?

Don't do this stupid party trick with `global`.

You may as well argue that unit tests are pointless because you could cheat by making the implementations return just the hardcoded values from the test cases.

Re: Architecture Patterns with Python

#136

Post by holdenweb »

Jcampuzano2 wrote: post #56
DanielVZ wrote: post #25

Wow this book is a goldmine for architecture patterns. I love how easy it is to get into a topic and quickly grasp it. Having said that, from a practical and experience standpoint, using some of these patterns can really spiral out into an increased complexity and performance issues in Python, specially when you use already opinionated frameworks like Django which already uses the ActiveRecord pattern. I’ve been in c…

I think one of the biggest problems I encounter whenever I hear that a project follows strict architectural patterns essentially boils down to too many obfuscated abstractions that hide what is going on, or force you to jump through too many layers to accomplish tasks. Many files/functions/classes need to be updated to accomplish even simple tasks because somebody made a decision that you aren't allowed to do X or Y…

I think you just discovered software engineering, which at its best makes intelligent tradeoffs to optimise use of resources to meet needs.

Re: Architecture Patterns with Python

#137

Post by pansa2 »

_dain_ wrote: post #135
pansa2 wrote: post #125

Earlier quoted context omitted.

> So don't do this then? Don't do what? - Don't write unsound code? There's no way to know until you run the program and find out your `int` is actually a `list`. - Don't assume type annotations are correct? Then what's the point of all the extra code to appease the type checker if it doesn't provide any guarantees?

Don't do this stupid party trick with `global`. You may as well argue that unit tests are pointless because you could cheat by making the implementations return just the hardcoded values from the test cases.

This isn't a "party trick" with `global`, it's a fundamental hole in the type system:

    class C:
        def __init__(self) -> None:
            self.i : int | list[int] = 0

        def foo(self) -> None:
            self.i = []
        
        def bar(self) -> int:
            if isinstance(self.i, int):
                self.foo()
                return self.i
            return 0

    print(type(C().bar()))

Re: Architecture Patterns with Python

#138

Post by zahlman »

_dain_ wrote: post #134
zahlman wrote: post #128

Earlier quoted context omitted.

>No, you do gain information from it: that the function takes an Iterable[Ducklike]. I already had that information. I understand my own coding style. >Being able to reason about a function without reading its implementation is a straightforward win. My function bodies are generally only a few lines, but my reasoning here is based on the choice of identifier name. Yes, it takes discipline, but it's the same kind of d…

>I already had that information. I understand my own coding style. Good for you, but you're not the only person working on the codebase, surely. >My function bodies are generally only a few lines, but my reasoning here is based on the choice of identifier name. Your short functions still call other functions which call other functions which call other functions. The type will not always be obvious from looking at the…

> Good for you, but you're not the only person working on the codebase, surely.

I actually am. But I've also read plenty of non-annotated Python code from strangers without issue. Including the standard library, random GitHub projects I gave a PR to fix some unidiomatic expression (defense in depth by avoiding `eval` for example), etc. When the code of others is type-annotated, I often find it just as distracting as all the "# noqa: whatever" noise not designed to be read by humans.

And long functions are vastly more mentally taxing.

> often all a function does with an argument is forward it along untouched to another function. You often still need to jump through many layers of the call graph to figure out how something actually gets used.

Yes, and I find from many years of personal experience that this doesn't cause a problem. I don't need to "figure out how something actually gets used" in order to understand the code. That's the point of organizing it this way. This is also one of the core lessons of SICP as I understood it. The dynamic typing of LISP is not an accident.

> An identifier name can't be as expressive as a type without sacrificing concision

On the contrary: it is not restricted to referring to abstractions that were explicitly defined elsewhere.

> Why not be precise, why not offload some mental work onto the computer?

When I have tried to do it, I have found that the mental work increased.

> No, see, this is an absolutely crucial point of disagreement

It is.

Re: Architecture Patterns with Python

#139

Post by lazyasciiart »

unculture wrote: post #24
kelafoja wrote: post #16

Earlier quoted context omitted.

Could you explain how repository pattern is a "huge overkill that adds complexity with very little benefit"? I find it a very light-weight pattern and would recommend to always use it when database access is needed, to clearly separate concerns. In the end, it's just making sure that all database access for a specific entity all goes through one point (the repository for that entity). Inside the repository, you can d…

Repository pattern is useful if you really feel like you're going to need to switch out your database layer for something else at some point in the future, but I've literally never seen this happen in my career ever. Otherwise, it's just duplicate code you have to write.

I’ve seen it, but of course there was no strict enforcement of the pattern so it was a nightmare of leakage and the change got stuck half implemented, with two databases in use.

Re: Architecture Patterns with Python

#140

Post by bulatb »

psd1 wrote: post #51
kelafoja wrote: post #46

Earlier quoted context omitted.

Thanks, that makes a lot of sense. I don't have a whole bunch of experience with SQLAlchemy itself. In general, I prefer not to use ORMs but just write queries and map the results into value objects. That work I would put into a Repository. Also in my opinion it's important to decouple the database structure from the domain model in the code. One might have a Person type which is constructed by getting data from 3 ta…

I've used SqlAlchemy in a biggish project. Had many problems, the worst ones were around session scoping and DB hitting season limits, but we had issues around the models too. The argument for hiding SqlAlchemy is nothing to do with "what if we change the DB"; that's done approximately never, and, even if so, you have some work to do, so do it at the time. YAGNI The argument is that SA models are funky things with la…

Just a heads-up if you haven't seen it: Overriding lazy-loading options at query time can help with overfetching.

    class Author(Model):
        books = relationship(..., lazy='select')

    fetch_authors = select(Author).options(raiseload(Author.books))
Anything that gets its Authors with fetch_authors will get instances that raise instead of doing a SELECT for the books. You can throw that in a smoke test and see if there's anything sneaking a query. Or if you know you never want to lazy-load, relationship(..., lazy='raise') will stop it at the source.

https://docs.sqlalchemy.org/en/20/orm/queryguide/relationshi...

Post reply on HN