181–190 of 376 posts
Earlier quoted context omitted.
That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.
Genuine question: say you have 3-4 services and a bunch of databases that make up your product, what's the alternative to plemping them all into K8s according to you?
Earlier quoted context omitted.
That doesn’t sound like hiring the only „brightest“.
But we only hire from MIT, Waterloo, CMU, etc! No, truly, I feel you. The Waterloo kids are literally cut from a different cloth though I'll take one of them to ten of anyone else. I feel really guilty about grabbing young grads with all that potential and putting them through this... Grads aren't expected to know anything. It's all the early ex-google hiring that fucked us.
And for a consulting company, a solid message-based deployment is not a good business strategy. If things just work and temporary load spikes get buffered automatically, there's very little reason for clients to buy a maintenance retainer.
Earlier quoted context omitted.
To your comment on Airflow, I’ve been around that block a few times. I’ve found Airflow (and really any orchestration) be the most manageable when it’s nearly devoid of all logic to the point of DAGs being little more than a series of function or API calls, with each of those responsible for managing state transfer to the next call (as opposed to relying on orchestration to do so). For example, you need some ETL to h…
Helpful comment! If I could pick your brain... I'm looking at a green field implementation of a task system, for human tasks - people need to do a thing, and then mark that they've done it, and that "unlocks" subsequent human tasks, and near as I can tell the overall task flow is a DAG. I'm currently considering how (if?) to allow for complex logic about things like which tasks are present in the overall DAG - things…
Earlier quoted context omitted.
But we only hire from MIT, Waterloo, CMU, etc! No, truly, I feel you. The Waterloo kids are literally cut from a different cloth though I'll take one of them to ten of anyone else. I feel really guilty about grabbing young grads with all that potential and putting them through this... Grads aren't expected to know anything. It's all the early ex-google hiring that fucked us.
Not sure how big your company is and how large your influence. But it sounds to me like your new hires would benefit from more guidance and oversight. Those mistakes shouldn’t make it production unless nobody gives a damn.
I do get to slam my foot down on some things from time to time -- usually for compliance reasons. Like now when people tell me they need credentials for things so they can run their code in dev on their laptops, I tell them no and that they can use mocks for their testing like a professional.
Funnily the junior people expect and have no problem with this -- it's the seasoned folks who chafe and complain.
Earlier quoted context omitted.
We also have much much bigger single machines available for reasonable money. So a lot of reasonable workloads can fit in one machine now that used to require a small cluster
It's kind of mind boggling just how powerful mundane desktop computers have gotten, let alone server hardware. Think about it: That 20 core CPU (eg: i7 14700K) you can buy for just a couple hundred dollars today would have been supercomputer hardware costing tens or hundreds of thousands of dollars just a decade ago.
Earlier quoted context omitted.
I'm sure this happens. But ... most websites I load up have like a dozen things trying to gather data, whether for tracking, visitor analytics, observability, etc. Every time I view a page, multiple new unimportant messages are being sent out, and presumably processed asynchronously. Every time I order something, after I get the order confirmation page, I get an email and possibly a text message, both of which should…
Bingo - I work on the backend of a medical system and basically anything that interacts with a 3rd party gets put into a queue so our application doesn't choke immediately when one of them has issues. We also have some uses for it within our system. As far as the question, I was thinking that queues have probably just become a standard aspect of modern distributed systems; it's considered a pretty foundational cloud…
Earlier quoted context omitted.
That‘s nice, but the question is if you (i.e. your company) needs this ecosystem.
Genuine question: say you have 3-4 services and a bunch of databases that make up your product, what's the alternative to plemping them all into K8s according to you?
This way, Azure does most of the heavy lifting you would otherwise have to do yourself, even with managed kubernetes.
I can offer one data point. This is from purely startup-based experience (seed to Series A). A while ago I moved from microservices to monolith because they were too complicated and had a lot of duplicated code. Without microservices there's less need for a message queue. For async stuff, I used RabbitMQ for one project, but it just felt...old and over-architected? And a lot of the tooling around it (celery) just was…
In my experience monoliths don't reduce complexity, they just shift it. The main issue with monoliths is that they don't have clear and explicit separation of concern between domain concerns, therefore it's very easy for your monolith codebase to devolve into a mess of highly interconnected spaghetti code with time. This is especially true if you're building something large with a lot of developers who don't necessar…
This is both true and false in a way. Sure, the same business logic is distributed across microservices, but a method call in a monolith can only fail in a couple of ways, while network calls are much more finicky - handling it in every case is pure added complexity in case of a microservice architecture.
Also, don’t forget the observability part - a mainstream language will likely have a sane debugger, profiler, a single log stream, etc. I can easily find bugs, race conditions, slow code paths in a monolith. It’s much more difficult if you have to do it in a whole environment communicating with potentially multiple instances of a single, or multiple microservices.
Lastly, we have programming languages to help us write correct, maintainable code! A monolith != spaghetti code. We have language tools to enforce boundaries, we have static analysis, etc. A refactor will work correctly across the whole codebase. We have nothing of this sort for microservices. You might understand a given microservice better, but does anyone understand the whole graph of them? Sure, monoliths might become spaghettis, but microservices can become spaghettis that are tangled with other plates of spaghettis.