Having run into these issues with Pydantic, we've been using Mashumaro[1], which, while not having all the bells and whistles of Pydantic, has served us pretty well. 1: https://github.com/Fatal1ty/mashumaro
I use Attrs instead of Pydantic
31–40 of 76 posts
Re: I use Attrs instead of Pydantic
#32Specifically I use pydantic for validation, so yes, I expect pydantic to iterate over a list of 10000 elements and verify these are all ints.
It needs to be configurable IMO, but yeah for small lists I would rather loop and check than not check at all.
However doing this for arbitrary iterables is impossible, because the iterable might be lazy or infinite. In that case maybe the best option is to wrap the iterable with something that would validate each element as it came off of the iterator that is produced from the iterable.
Re: I use Attrs instead of Pydantic
#33Recently there's a strong tendency in the Python ecosystem to wrap perfectly fine idioms with even higher abstractions. Not sure if I like this, in the end it leads to a lot of cargo culting and people writing code they don't fully understand because it contains a lot of metaclass and runtime magic. But I understand that a lot of developers find this attractive, and type annotations really provide a good channel for…
Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone. The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.
I think the comment about types takes a rather narrow view of Python use cases. Sure, if you’re some lone sysadmin using Python as an alternative to bash to implement some questionable ETL pipeline comprising a bunch of random scripts running off a single machine somewhere then I agree. You probably are not the target audience for type annotations.
But even moderately complex libraries and applications are such a huge pain to develop, read, and maintain without some of the tooling that leverages type annotations. This is especially true in the ML / AI / Data Science world where a lot of the people implementing models have dubious coding practices.
Pure dynamic typing paradigms simply have not delivered on their promises over the last 30 years. There are definitely some areas where they make sense, but I doubt we will see a massive readoption until our tooling becomes sufficiently intelligent. Imagine, for example, a probabilistic type inference based on both the structural aspects of the code and previous runs over actual data.
Re: I use Attrs instead of Pydantic
#34That's just like, your opinion, man!
I'm a die-hard pydantic fan, but it's refreshing to see other perspectives, while the author is acknowledging it is their opinion, without getting all holy-war. Also, they aren't orthogonal. Pydantic is definitely heavier than attrs in terms of processing. This is what makes pydantic a great bastion at shearing layers. "Validate all 10000 ints" is exactly what you want when parsing a request, CLI input, or some configuration.
Also, pydantic makes it almost trivial to write top-level app config logic that is populated from configs, env variables, secrets, etc.
Also, I can generate pydantic structures from openApi, jsonschema, etc, and conversely generate schema/swagger from pydantic. This is game-breaking amounts of awesomeness.
On the other hand, inside business logic, pydantic has the downsides TFA mentions. I would however still contend some validation sprinkled in with business is helpful to reign in some of that zany python dynamicism in huge codebases.
I like the simplicity and composability of the c/attrs approach. Along with the "pydantic does not like positional args," (`__root__=(a,b,c,...)`, ugh) I'll be considering this for retrofitting some code paths that are currently dicts with actual structured types.
I didn't know about the performance differences. Sam Colvin strikes me as very thorough and the community is very involved, so I don't think the benchmarks claiming pydantic faster than attrs is wrong. I know pydantic uses cython under the hood (no idea what) - is it possible that environmental differences are causing the discrepancy?
Re: I use Attrs instead of Pydantic
#35That's just like, your opinion, man! I'm a die-hard pydantic fan, but it's refreshing to see other perspectives, while the author is acknowledging it is their opinion, without getting all holy-war. Also, they aren't orthogonal. Pydantic is definitely heavier than attrs in terms of processing. This is what makes pydantic a great bastion at shearing layers. "Validate all 10000 ints" is exactly what you want when parsin…
As of 2021, there is now a complete Pydantic "CRUD stack" (ODMantic/SQLModel + FastAPI) that is ridiculously easy to get up and running with, and it has all of the features one might want in a brave new type safe world.
By comparison, the system of tools around Attrs and Cattrs is a bit scattered and underdeveloped. I don't think there's a technical reason for it, that's just how it happened. It would be great to see either a competitive parallel "stack" developing, or increased interoperability from libraries that currently only support one or the other.
For this reason, in my personal projects I use Attrs almost all the time when writing new classes. I don't use Pydantic. But at work, we are very likely going to adopt the aforementioned "Pydantic CRUD stack" for new internal services, specifically because our needs are straightforward, and for straightforward use case is a "just works" and "there's only one right way to do it", which I think are extremely valuable features when operating in a team.
Re: I use Attrs instead of Pydantic
#36Recently there's a strong tendency in the Python ecosystem to wrap perfectly fine idioms with even higher abstractions. Not sure if I like this, in the end it leads to a lot of cargo culting and people writing code they don't fully understand because it contains a lot of metaclass and runtime magic. But I understand that a lot of developers find this attractive, and type annotations really provide a good channel for…
Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone. The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.
Static typing in python isn't "cargo culting." In fact I'd go even farther in the opposite direction, and say that if you are using python for production software engineering, and aren't typing most (>50%) of your function signatures, you are wasting everybody's time, your teams', code reviewers', and most of all your own time. Opinion, but empirical experience bears this out for me.
I'm not talking mypy --strict here, heck I've yet to achieve that myself. But if your are writing functions with obvious realizations of interfaces and not typing them, all you are doing is creating more mental strain for the next consumer (often future you) of that function down the road.
At $LASTCO I inherited some jupyter-notebook-copy-pasted datascience ML abomination of a pipeline. Dicts everywhere, mutation everywhere, zero docstrings, nary a type hint in sight. Took two exceptionally stressful months, with constant back and forth with the authors, to get it working in prod ("it works on my machine, I don't understand the problem"). $THISCO embraces type hints, functional style, etc. My stress level has actually normalized.
Typing `foo(bar: Optional[float])` takes what, 2s, more than `foo(bar)`? Asking "hey Steve, what does "bar" take in function "foo"? on slack is already more time and characters than just annotating it, times every dev that doesn't know and has to ask.
Re: I use Attrs instead of Pydantic
#37That's just like, your opinion, man! I'm a die-hard pydantic fan, but it's refreshing to see other perspectives, while the author is acknowledging it is their opinion, without getting all holy-war. Also, they aren't orthogonal. Pydantic is definitely heavier than attrs in terms of processing. This is what makes pydantic a great bastion at shearing layers. "Validate all 10000 ints" is exactly what you want when parsin…
As of 2021, there is now a complete Pydantic "CRUD stack" (ODMantic/SQLModel + FastAPI) that is ridiculously easy to get up and running with, and it has all of the features one might want in a brave new type safe world. By comparison, the system of tools around Attrs and Cattrs is a bit scattered and underdeveloped. I don't think there's a technical reason for it, that's just how it happened. It would be great to see…
What's the use case for Attrs in a post-dataclasses world? I use dataclasses all the time, and I've never felt the need to reach for Attrs. What am I missing out on?
Re: I use Attrs instead of Pydantic
#38Earlier quoted context omitted.
As of 2021, there is now a complete Pydantic "CRUD stack" (ODMantic/SQLModel + FastAPI) that is ridiculously easy to get up and running with, and it has all of the features one might want in a brave new type safe world. By comparison, the system of tools around Attrs and Cattrs is a bit scattered and underdeveloped. I don't think there's a technical reason for it, that's just how it happened. It would be great to see…
What's the use case for Attrs in a post-dataclasses world? I use dataclasses all the time, and I've never felt the need to reach for Attrs. What am I missing out on?
More configurability, slots by default, faster to create the class object. I've heard stories that loading a module with a lot of dataclasses defined in it can be really slow. Basically dataclasses are for the people who can't afford a 3rd-party dependency for whatever reason.
Re: I use Attrs instead of Pydantic
#39Earlier quoted context omitted.
Yes, this higher abstractions tendency is clearly away from "Flat is better than nested". Creating good abstractions is so difficult (and important) that it should not be done lightly by everyone. The recent proliferation of type annotations is also trending towards the static typing mindset, which is also cargo culting on a massive scale.
Static typing in python isn't "cargo culting." In fact I'd go even farther in the opposite direction, and say that if you are using python for production software engineering, and aren't typing most (>50%) of your function signatures, you are wasting everybody's time, your teams', code reviewers', and most of all your own time. Opinion, but empirical experience bears this out for me. I'm not talking mypy --strict her…
Type hints just make everything so much easier. When you're writing a function, you _know_ that parameter is an int, or float, or something you can use a list comprehension on (iterable). Why not just say it right there in the function definition?
It's better than writing it in a docstring, because a type checker will tell you to change the type if you change how you use a variable.
Does everyone need to go all the way and type 100% of things and use heavily generic code to represent all possible cases? Well, that would be wonderful, but just sprinkling built-in types is already a massive improvement over no types at all.
Re: I use Attrs instead of Pydantic
#40From my understanding, size: float = None is syntactic sugar for size: Optional[float]
Mypy allows that because initial versions of PEP-484 allowed that. This has changed; here's the current wording on the PEP:
> This is no longer the recommended behavior. Type checkers should move towards requiring the optional type to be made explicit.