Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Architecture Patterns with Python

cosmicpython.com

Post reply on HN
Print view View original

51–60 of 143 posts

Re: Architecture Patterns with Python

#51

Post by psd1 »

kelafoja wrote: post #46
bulatb wrote: post #40

Earlier quoted context omitted.

In theory it's a nice abstraction, and the benefit is clear. In practice, your repository likely ends up forwarding its arguments one-for-one to SQLAlchemy's select() or session.query(). That's aside from their particular example of SQLAlchemy sessions, which is extra weird because a Session is already a repository, more or less. I mean, sure, there's a difference between your repository for your things and types you…

Thanks, that makes a lot of sense. I don't have a whole bunch of experience with SQLAlchemy itself. In general, I prefer not to use ORMs but just write queries and map the results into value objects. That work I would put into a Repository. Also in my opinion it's important to decouple the database structure from the domain model in the code. One might have a Person type which is constructed by getting data from 3 ta…

I've used SqlAlchemy in a biggish project. Had many problems, the worst ones were around session scoping and DB hitting season limits, but we had issues around the models too.

The argument for hiding SqlAlchemy is nothing to do with "what if we change the DB"; that's done approximately never, and, even if so, you have some work to do, so do it at the time. YAGNI

The argument is that SA models are funky things with lazy loading. IIRC, that's the library where the metaclasses have metaclasses! It's possible to accidentally call the DB just by accessing a property.

It can be a debugging nightmare. You can have data races. I remember shouting at the code, "I've refreshed the session you stupid @#£*"

The responsible thing to do is flatten them to, say, a pydantic DTO. Then you can chuck them about willy-nilly. Your type checker will highlight a DTO problem that an SA model would have slipped underneath your nose.

The difficulty you have following that is that, when you have nested models, you need to know in advance what fields you want so you don't overfetch. I guess you're thinking "duh, I handcraft my queries" and my goodness I see the value of that approach now. However, SA still offers benefits even if you're doing this more tightly-circumscribed fetch-then-translate approach.

This is partly how I got from the eager junior code golf attitude to my current view, which is, DO repeat yourself, copy-paste a million fields if you need, don't sweat brevity, just make a bunch of very boring data classes.

Re: Architecture Patterns with Python

#52

Post by abirch »

exe34 wrote: post #50
DeathArrow wrote: post #45

Earlier quoted context omitted.

I am not convinced that domain driven design works. Objects doesn't model the real world well. Why we should think DDD model the real world or a business well? And why do we even need to model something? Computers are different than humans. I think we should be pragmatic and come with the best solution in terms of money/time/complexity. Not trying to mimick human thought using computers. After all a truck isn't mimic…

the model is for me. I wrote a crude btc trading bot, and I had three major objects: Market(), TradingStrategy and SimulatedMarket which used historical data. I did not model individual transactions, fees, etc as objects. The idea was that I encapsulated all the stuff that's relevant to a strategy in one object, and the market object would give me things like get_price(), buy(), sell(), which would also be available…

And if you can encapsulate the 3 different domains well, if you switch brokers you should have any external dependencies. Make your functions/domains deep.

Re: Architecture Patterns with Python

#53

Post by BerislavLopac »

kelafoja wrote: post #43
BerislavLopac wrote: post #42

Earlier quoted context omitted.

OK, let's first define some things. What we are talking about is a "transformation" or "mapper" layer isolating your domain entities from the persistence. If this is what we call "Repository" then yes, I absolutely agree with you -- this is the right approach to this problem. But if the "Repository pattern" means a complex structure of abstract and concrete classes and inheritance trees -- as I have usually seen it i…

Thanks. In my mind, anything about complex structures of (abstract) classes and/or inheritance trees has nothing to do with a Repository pattern. As I understand it, Repository pattern is basically a generalization of the Data Access Object (DAO) pattern, and sometimes treated synonymously. The way I mean it and implement it, is basically for each entity have a separate class to provide the database access. E.g. you…

> for each entity have a separate class to provide the database access

Let me correct you: for each entity that needs database access. This is why I'm talking about layers here: sometimes entities are never persisted directly, but only as "parts" or "relations" of other entities; in other cases you might have a very complex persistence implementation (e.g. some entities are stored in a RDB, while others in a filesystem) and there is no clear mapping.

I recommend you to approach this from the perspective of each domain entity individually; "persistability" is essentially just another property which might or might not apply in each case.

Re: Architecture Patterns with Python

#54

Post by globular-toast »

DeathArrow wrote: post #45
exe34 wrote: post #38

Earlier quoted context omitted.

my favourite model is to write as many pure functions as possible, and then as many functions of 1-4 parameters that interact with the outside world, and only then create domain objects to wrap those - it keeps the unrelated complexity out of the domain and then I can also reuse those functions without having to create the entire object that I don't always need.

I am not convinced that domain driven design works. Objects doesn't model the real world well. Why we should think DDD model the real world or a business well? And why do we even need to model something? Computers are different than humans. I think we should be pragmatic and come with the best solution in terms of money/time/complexity. Not trying to mimick human thought using computers. After all a truck isn't mimic…

DDD isn't about objects. It's just about modelling the domain (real world) using the tools available to you. Some things are best modelled by objects, some are best modelled by functions or other constructs.

The real point is establish a common language to talk about the domain. This is enormously powerful. Have you ever worked with people who don't speak your language? Everything takes 3x as long as ideas aren't communicated properly and things get lost in translation.

It's the same with code. If your code is all written in the language of the computer then you'll be translating business language into computer language. This takes longer and it's more error prone. The business people can't check your work because they don't know how to code and the coders can't check because they don't know the business.

The point of building abstractions is it empowers you to write code in a language that is much closer to the language of the domain experts. DDD is one take but the basic idea goes all the way back to things like SICP.

Re: Architecture Patterns with Python

#55

Post by dlisboa »

Pandabob wrote: post #4

Truly one of the great python programming books. The one thing that I found missing was the lack of static typing in the code, but that was a deliberate decision by the authors.

> The one thing that I found missing was the lack of static typing in the code

It has type hints, such as here: https://www.cosmicpython.com/book/chapter_08_events_and_mess...

Do you mean it's not strict enough? There are some parts of the book without them.

Re: Architecture Patterns with Python

#56

Post by Jcampuzano2 »

DanielVZ wrote: post #25

Wow this book is a goldmine for architecture patterns. I love how easy it is to get into a topic and quickly grasp it. Having said that, from a practical and experience standpoint, using some of these patterns can really spiral out into an increased complexity and performance issues in Python, specially when you use already opinionated frameworks like Django which already uses the ActiveRecord pattern. I’ve been in c…

I think one of the biggest problems I encounter whenever I hear that a project follows strict architectural patterns essentially boils down to too many obfuscated abstractions that hide what is going on, or force you to jump through too many layers to accomplish tasks.

Many files/functions/classes need to be updated to accomplish even simple tasks because somebody made a decision that you aren't allowed to do X or Y thing without creating N other things.

But in those companies that didn't care about architectural patterns its very likely that while there was more ugly code in certain places, it resulted in code with less indirection and more contained to a single area/unit or the task at hand making it easier for people to jump in and understand. I see so many people who create function after function in file after file to abstract away functionality when I'd honestly rather have a 100 line function or method that I can easily jump around and edit/debug vs many tiny functions all in separate areas.

Not to say having some abstractions are bad but the more I work in this field the more I realize the less abstractions there are, the easier it is to reason about singular units/features in code. I've basically landed on just abstract away the really hard stuff, but stop abstracting out things that simple.

Re: Architecture Patterns with Python

#57

Post by tayo42 »

natdempk wrote: post #26
pmg101 wrote: post #22

Earlier quoted context omitted.

My experience is that once people have static typing to lean on they focus much less on the things that in my view are more crucial to building clean, readable code: good, consistent naming and small chunks. Just the visual clutter of adding type annotations can make the code flow less immediately clear and then due to broken windows syndrome people naturally care less and less about visual clarity.

So far off from what actually happens. The type annotations provide an easy scaffolding for understand what the code does in detail when reading making code flow and logic less ambiguous. Reading Python functions in isolation, you might not even know what data/structure you’re getting as input… if there’s something that muddles up immediate clarity it’s ambiguity about what data code is operating on.

Type annotations are just like documentation though. Just because the annotation says int the function can still return a list.

Re: Architecture Patterns with Python

#58

Post by bulatb »

kelafoja wrote: post #46
bulatb wrote: post #40

Earlier quoted context omitted.

In theory it's a nice abstraction, and the benefit is clear. In practice, your repository likely ends up forwarding its arguments one-for-one to SQLAlchemy's select() or session.query(). That's aside from their particular example of SQLAlchemy sessions, which is extra weird because a Session is already a repository, more or less. I mean, sure, there's a difference between your repository for your things and types you…

Thanks, that makes a lot of sense. I don't have a whole bunch of experience with SQLAlchemy itself. In general, I prefer not to use ORMs but just write queries and map the results into value objects. That work I would put into a Repository. Also in my opinion it's important to decouple the database structure from the domain model in the code. One might have a Person type which is constructed by getting data from 3 ta…

I mean that from the point of view of YAGNI for a small app. For a big one, absolutely, you will find the places where the theoretical distinctions suddenly turn real. Decoupling your data model from your storage is a real concern and Session on its own won't give you that advantage of a real repository layer.

SQLAlchemy is flexible, though. You can map a Person from three tables if you need to. It's a data mapper, then a separate query builder on top, then a separate ORM on top of that, and then Declarative which ties them all together with an ActiveRecord-ish approach.

> I prefer not to use ORMs but just write queries and map the results into value objects. That work I would put into a Repository.

Yep, I hear ya. Maybe if they'd built on top of something lower-level like stdlib sqlite3, it wouldn't be so tempting to dismiss as YAGNI. I think my comment sounded more dismissive than I really meant.

Re: Architecture Patterns with Python

#59

Post by pbronez »

psd1 wrote: post #51
kelafoja wrote: post #46

Earlier quoted context omitted.

Thanks, that makes a lot of sense. I don't have a whole bunch of experience with SQLAlchemy itself. In general, I prefer not to use ORMs but just write queries and map the results into value objects. That work I would put into a Repository. Also in my opinion it's important to decouple the database structure from the domain model in the code. One might have a Person type which is constructed by getting data from 3 ta…

I've used SqlAlchemy in a biggish project. Had many problems, the worst ones were around session scoping and DB hitting season limits, but we had issues around the models too. The argument for hiding SqlAlchemy is nothing to do with "what if we change the DB"; that's done approximately never, and, even if so, you have some work to do, so do it at the time. YAGNI The argument is that SA models are funky things with la…

Based on that, do you find SQLModel[0] to be an elegant integration of these ideas, or a horrid ball of spaghetti?

[0] https://sqlmodel.tiangolo.com/

Re: Architecture Patterns with Python

#60

Post by globular-toast »

bulatb wrote: post #40
kelafoja wrote: post #16

Earlier quoted context omitted.

Could you explain how repository pattern is a "huge overkill that adds complexity with very little benefit"? I find it a very light-weight pattern and would recommend to always use it when database access is needed, to clearly separate concerns. In the end, it's just making sure that all database access for a specific entity all goes through one point (the repository for that entity). Inside the repository, you can d…

In theory it's a nice abstraction, and the benefit is clear. In practice, your repository likely ends up forwarding its arguments one-for-one to SQLAlchemy's select() or session.query(). That's aside from their particular example of SQLAlchemy sessions, which is extra weird because a Session is already a repository, more or less. I mean, sure, there's a difference between your repository for your things and types you…

SQLAlchemy Session is actually a unit of work (UoW), which they also build on top. By the end of the book they are using their UoW to collect and dispatch events emitted by the services. How would they have done that if they just used SQLAlchemy directly?

You might argue that they should have waited until they wanted their own UoW behaviour before actually implementing it, but that means by the time they need it they need to go and modify potentially hundreds of bits of calling code to swap out SQLAlchemy for their own wrapper. Why not just build it first? The worst that happens is it sits there being mostly redundant. There have been far worse things.

The tricks you mention for the tests might work for SQLAlchemy, but what if we're not using SQLAlchemy? The repository pattern works for everything. That's what makes it a pattern.

Post reply on HN