Resume driven design, is running into the desert of moores plateau punishing the use of ever more useless abstractions. They get quieter, because their projects keep on dying after the revolutionary tech is introduced and they jump ship.
Kafka is Fast – I'll use Postgres
11–20 of 412 posts
Re: Kafka is Fast – I'll use Postgres
#12[deleted]
Re: Kafka is Fast – I'll use Postgres
#13You have to be careful with the approach of using Postgres for everything. The way it locks tables and rows and the serialization levels it guarantees are not immediately obvious to a lot of folks and can become a serious bottle-neck for performance-sensitive workloads. I've been a happy Postgres user for several decades. Postgres can do a lot! But like anything, don't rely on maxims to do your engineering for you.
Yes, performance can be a big issue with postgres. And vertical scaling can really put a damper on things when you have a major traffic hit. Using it for kafka is misunderstanding the one of the great uses of kafka which is to help deal with traffic bursts. All of a sudden your postgres server is overwhelmed and the kafka server would be fine.
Re: Kafka is Fast – I'll use Postgres
#14> Should You Use Postgres? Most of the time - yes. You should always default to Postgres until the constraints prove you wrong. Kafka, GraphQL... These are the two technology's where my first question is always this: Does the person who championed/lead this project still work here? The answer is almost always "no, they got a new job after we launched". Resume Architecture is a real thing. Meanwhile the people left be…
We’re all passing through our jobs, the value of the solutions remains in the hands of the shareholders, if you don’t try to squeeze some long-term value for your resume and long-term employability, you’re assuming a significant opportunity cost on their behalf
They’ll be fine if you made something that works, even if it was a bit faddish, make sure you take care of yourself along the way (they won’t)
Re: Kafka is Fast – I'll use Postgres
#15> One camp chases buzzwords > ... > The other camp chases common sense I don't really like these simplifications. Like one group obviously isn't just dumb, they're doing things for reasons you maybe don't understand. I don't know enough about data science to make a call, but I'm guessing there were reasons to use Kafka due to current hardware limits or scalability concerns, and while the issues may not be as present…
Kafka and other message systems like it have their uses. But sometimes all you need is just need a database. Now you start doing realtime streaming and notifications and event type things a messaging system is good. You can even back it up with a boring database. Would I start with kafka? Probably not. I would start with a boring databsee and then if if my bashing on the db over and over saying 'have you changed' doesnt work as good anymore then you put in a messaging system.
Re: Kafka is Fast – I'll use Postgres
#16For me the killer feature of Kafka was the ability to set the offset independently for each consumer.
In my company most of our topics need to be consumed by more than one application/team, so this feature is a must have. Also, the ability to move the offset backwards or forwards programmatically has been a life saver many times.
Does Postgres support this functionality for their queues?
Re: Kafka is Fast – I'll use Postgres
#17I'm starting to like mongodb a lot more given the python library mongomock. I find it wonderful to create tests that run my queries against mongo in code before I deploy them. Yes, mongo has a lot of quirks and you have to know aws networking to set it up with your vpc so you don't get nailed with egress costs. And it's not the same query patterns and some queries are harder and you have maintain your own schemas. Bu…
Or just use devcontainers and have an actual Postgres DB to test against? I've even done this on a Chromebook. This is a solved problem.
Re: Kafka is Fast – I'll use Postgres
#18I'm starting to like mongodb a lot more given the python library mongomock. I find it wonderful to create tests that run my queries against mongo in code before I deploy them. Yes, mongo has a lot of quirks and you have to know aws networking to set it up with your vpc so you don't get nailed with egress costs. And it's not the same query patterns and some queries are harder and you have maintain your own schemas. Bu…
Curious why you think the risk of edge cases from mocking is a worthwhile trade off vs the relatively low complexity of setting up a container to test against?
Re: Kafka is Fast – I'll use Postgres
#19> Should You Use Postgres? Most of the time - yes. You should always default to Postgres until the constraints prove you wrong. Kafka, GraphQL... These are the two technology's where my first question is always this: Does the person who championed/lead this project still work here? The answer is almost always "no, they got a new job after we launched". Resume Architecture is a real thing. Meanwhile the people left be…
"resume" as in "resumé" not as in "begin again or continue after a pause or interruption" - it took me longer than I care to admit to get that.
Re: Kafka is Fast – I'll use Postgres
#20> Should You Use Postgres? Most of the time - yes. You should always default to Postgres until the constraints prove you wrong. Kafka, GraphQL... These are the two technology's where my first question is always this: Does the person who championed/lead this project still work here? The answer is almost always "no, they got a new job after we launched". Resume Architecture is a real thing. Meanwhile the people left be…
We’re all passing through our jobs, the value of the solutions remains in the hands of the shareholders, if you don’t try to squeeze some long-term value for your resume and long-term employability, you’re assuming a significant opportunity cost on their behalf They’ll be fine if you made something that works, even if it was a bit faddish, make sure you take care of yourself along the way (they won’t)
Attitudes like this are why management treats developers like children who constantly need to be kept on task, IMO.