231–240 of 376 posts
This but also: computers got incredibly more capable. You can now have several terabytes of ram and literally hundreds of cpu cores in a single box.
Chances are you can take queuing off your design.
Earlier quoted context omitted.
> And then there is Helm Right, who doesn't want to template hundreds of lines of code in a language that uses whitespace for logic and was never made neither for templating nor complex long documents(YAML)? What could possibly go wrong ("error missing xxx at line 728, but it might be a problem elsewhere").
"in a language that uses whitespace for logic" This argument kind of died when python became one of the most popular programming languages used.
Going to give the unpopular answer. Queues, Streams and Pub/Sub are poorly understood concepts by most engineers. They don't know when they need them, don't know how to use them properly and choose to use them for the wrong things. I still work with all of the above (SQS/SNS/RabbitMQ/Kafka/Google Pub/Sub). I work at a company that only hires the best and brightest engineers from the top 3-4 schools in North America a…
That doesn’t sound like hiring the only „brightest“.
Going to give the unpopular answer. Queues, Streams and Pub/Sub are poorly understood concepts by most engineers. They don't know when they need them, don't know how to use them properly and choose to use them for the wrong things. I still work with all of the above (SQS/SNS/RabbitMQ/Kafka/Google Pub/Sub). I work at a company that only hires the best and brightest engineers from the top 3-4 schools in North America a…
« Use kafka as a basic message queue ». Since i’m guilty of that (i use kafka as the backbone for pretty much any service 2 service communication, under a « job » api) i wonder why you think that’s wrong.
Be careful not to conflate message transport with message storage, although message brokers usually do both.
SQS was slower than frozen molasses when I used it c. 2010. ZMQ, ZK, rabbit, and MQTT are oft mentioned. ESBs always come up and a large % of technical people hate them because they come with "software architect" historical baggage.
It's risky to have a SPoF one-grand-system-to-rule-them-all when you can have a standardized API for every department or m[ia]cro-service exposed as RESTful, gRPC, and/or GraphQL in a more isolated manner.
Redis isn't needed in some ecosystems like Elixir/Erlang/BEAM. Memcache is simpler if you just need a temporary cache without persistence.
One project I know of started with message queues for request response pattern. It performed poorly because Windows Service Bus writes messages to a database. That increased latency for a UI heavy application.
Second project used message queues but the front end was a HTTP API. When overloaded the API timed out at 30 seconds but the job was still in the queue and wasn’t cancelled. It led to a lot of wastage.
Going to give the unpopular answer. Queues, Streams and Pub/Sub are poorly understood concepts by most engineers. They don't know when they need them, don't know how to use them properly and choose to use them for the wrong things. I still work with all of the above (SQS/SNS/RabbitMQ/Kafka/Google Pub/Sub). I work at a company that only hires the best and brightest engineers from the top 3-4 schools in North America a…
Are you making this up to make your claim sound more extravagant? Even Citadel is not that picky so unless you work at OpenAI/Anthropic I'm calling nonsense.
Basically everything in tech has gone through a hype cycle when everyone was talking about it, when it was the shiny new hammer that needed to be applied to every problem and appear in every resume.
A bit over 20 years ago I interviewed with a company who was hiring someone to help them "move everything to XML". Over the course of the two hour interview I tired unsuccessfully to figure out what they actually wanted to do. I don't think they actually understood what XML was, but I still wonder from time to time what they were actually trying to achieve and if they ever accomplished it.
We have a new project (~ 6 years now) where we implemented a queue with RabbitMQ to temporarily store business events before they are stored in a database for reporting later. It's awesome! It absorbs the peaks, smoothes them out, acts as a buffer for when the database is down for upgrades, and I think over all these years we only had one small issue with it. 10/10 would recommend.
Going to give the unpopular answer. Queues, Streams and Pub/Sub are poorly understood concepts by most engineers. They don't know when they need them, don't know how to use them properly and choose to use them for the wrong things. I still work with all of the above (SQS/SNS/RabbitMQ/Kafka/Google Pub/Sub). I work at a company that only hires the best and brightest engineers from the top 3-4 schools in North America a…
I am out of the loop here. Did Rabbit break classic queues?