161–170 of 213 posts
This whole image, to me, represents a big problem with software engineering today: https://twitter.com/dankohn1/status/989956137603747840 The industry is full of engineers who are experts in weirdly named "technologies" (which are really just products and libraries) but have no idea how the actual technologies (e.g. TCP/IP, file systems, memory hierarchy etc.) work. I don't know what to think when I meet engineers wh…
Earlier quoted context omitted.
If we're talking about the actual installation of a distributed system, then installing a distributed database isn't "easy" either, and also requires knowing the concepts so you know what you're doing. Kubernetes is not going to be simpler or easier than the software that it's designed to run on top of itself, but it's not that hard anymore either. The installers work well, there are several distros with varying capa…
Compare to setting up a Lambda application with a DynamoDB or Aurora database. Very simple, has limitations but you get a scalable distributed system for (almost) no time investment.
Yes, different things are different levels of complexity, and using a managed service means paying someone else to handle it.
Earlier quoted context omitted.
> for simple systems that clearly don't need it. unless you have a single server with no availability your system is probably not "simple".
We do have a hundred or so servers with multiple redundancies, but we've done okay without Kubernetes...
* what do you use for master election
* what do you use to deploy your stuff
* what do you use on the tcp level for HA (keepalived?)
* what do you use to keep your system up to date
these would be my first questions about your system. and if you would use k8s all of them could be answered with "kubernetes"
Basically agree that simple things should be simple and complex things possible - and that’s from my experience not yet met with standard k8s. And having no „end users“ (app developers) on a conference about tools that should serve exactly these is an interesting observation to investigate further. Having to install one mire tool to get ready for production apps installed with helm in one command is not asked too muc…
Earlier quoted context omitted.
There are specialist subfields of programming nowadays: frontend, mobile app, embedded/microcontroller. And a frontend sitebuilder doesen't have to know anything about transistors, FinFET, system calls, and only the most curious would know about reflow and painting calls in a browser's rendering engine. Sure, the quest for performance always pushes people toward the metal, but from a website's perspective that's just…
> And how bad the demand side is when it comes to picking the experts. You meant, there is a very strong demand ?
And yes, there's a demand for these specific skills, because big inefficient companies are willing to pay for just this, because their in house people are usually overworked or simply not allowed to do it themselves without an expert on site (or on call, or on skype).
And of course AWS is a big jungle of buzzwords itself, without clear documentation (and that's of course very important for them, they don't want to give away trade secrets, nor it should matter for the guarantees they offer, but their own custom nomenclature and sort of arbitrary abstractions over the standard Xen/KVM/qemu, VLAN, VXLAN, etc stuff makes it very hard to know exactly what's going on), so no surprise that there's demand for this. But as others said, sometimes these consultants have rather narrow skills.
And to add a bit more to this, generalists are few, and always quickly disappear from the market, so the holes/niches are plugged by whoever happens to be around.
Finally, I don't exactly blame these corps, it's just what makes sense nowadays. Need for extremely short turn around, instant risk-elimination, ASAP ROI, all point to "experts" and "consultants". No point in getting someone who understands the full stack top-to-bottom, when the project is in the initial phase and you just need EC2 and a DB on RDS.
The complexity of Kubernetes largely reflects the complexity of the problem. Nobody has delivered anything significantly simpler that hasn’t had a much smaller scope, and those tools approach the same level of complexity when composed with others to get the same level of functionality. But setting up Kubernetes is both well documented and automated on multiple cloud providers, whereas something like Nomad & Consul do…
> The complexity of Kubernetes largely reflects the complexity of the problem. I can solve most of these problems with a simple Linux + systemd in a much more readable way and am finished much quicker, even now that I am using k8s full time for 2 years. So, no. What it represents is the complexity of the community having multiple sources of requirements, and the complexity of the landscape with everybody trying to ma…
That being said, if you don’t need any of those things you probably don’t need Kubernetes either.
Basically agree that simple things should be simple and complex things possible - and that’s from my experience not yet met with standard k8s. And having no „end users“ (app developers) on a conference about tools that should serve exactly these is an interesting observation to investigate further. Having to install one mire tool to get ready for production apps installed with helm in one command is not asked too muc…
Author here. We have more work to do to make the happy path with the set of Influx tools (the TICK stack) more turn key and easier to run. The entire feature set is available in the open source versions. The thing we keep commercial is HA and scale out clustering (either we operate for you on AWS or you buy on-premise). Our work on 2.0 of the platform should make the happy path much easier, but software is a process…
Earlier quoted context omitted.
You skip over setting up Kubernetes. And keeping it running.
If we're talking about the actual installation of a distributed system, then installing a distributed database isn't "easy" either, and also requires knowing the concepts so you know what you're doing. Kubernetes is not going to be simpler or easier than the software that it's designed to run on top of itself, but it's not that hard anymore either. The installers work well, there are several distros with varying capa…
E.g. over years of using Etcd. I've come to the conclusion that none of the uses I've actually used it for were necessary, and so I've generally stopped using it. I'll use it again if I come across instances where consistency is sufficiently critical and the system needs to be distributed, but that's a rare situation to be in. E.g. I've seen lots of systems try to use it to distribute configuration data. Most of the time this doesn't need to be consistent, as long as you can determine whether or not it is (and so take outdated instances out of rotation, or not put them back in rotation). Same for load balancer configs, for example.
It's gone to the point where whenever someone mentions "distributed database" alarm bells goes off in my head. 9 out of 10 times it's a sign of someone over-engineering a system and building in unnecessary complexity.
What I want out of a system to manage my containers is something that reduces complexity of deployment and operation, not something that introduces more complexity. So far I've seen nothing of Kubernetes that indicates it fits that bill. Maybe it does on the very high end. My largest systems have "only" been in the few hundred containers/VM range over dozens of servers in 4-5 physical locations (on premises/co-location, managed and public cloud). For that scale, I've personally found Kubernetes overkill and adding too much complexity.
Maybe that'll change some day as it matures, but I'm not in a rush to complicate my stacks.
Earlier quoted context omitted.
Who are also heavily trained in the physiology of the organs they operate on, long before they're allowed anywhere near a scalpel. What you say about trades is true, but then we need to start making the distinction between software engineers and software technicians. I doubt that will go down well in an industry where everyone wants to be a senior software engineer, a tech lead or higher...
I'm thinking the physiology of the organs might be less complicated than understanding a modern applications at all levels of the stack. Maybe that would be understanding the organs, the underlying physical processes those organs regulate, the chemicals that are released as parts of those processes, and the effects and composition of those chemicals, and if it is possible to induce similar effects through replacement…
Biology is not much different, though there schooling seems to do a better job at exposing the student to all the layers we know about.
Earlier quoted context omitted.
I don't know what to think when I meet engineers who know how to setup an ELB on AWS but don't quite understand what a socket is... I keep meeting people who list say Postgres as a skill and when probed they admit they just clicked a button in Amazon RDS and that’s all they know about it. At some point it crosses the line into outright deception. This is the other side of the coin on why interviewing is so broken too…
I'd say this goes both ways. You have companies asking for the impossible that won't interview you otherwise, this will create an incentive to deceive. I'd say HR is at fault here for asking for 10 years experience in everything, instead of just looking for software engineers that have good fundamentals and can learn. I once applied for a job writing tools for embedded programmers that said they want experience in An…