My experience is that Kubernetes is too complex for the average functioning product. At our company, everyone is obsessed with it because it promises no cloud vendor lock ins! But at what cost? The complexity. Also the direction cloud vendors are going in my opinion, is more hardware-centric (eg. TPUs). How will you avoid the cloud lock in when only Azure offers the image tagging machine learning as a service? How will Kubernetes solve that? I believe a balance between a small bit of lock-in, but retain environment freedom (free programming languages like python, javascript...) is the sweet spot for cloud, eg. PaaS like appengine or azure app service or beanstalk.
You might not need Kubernetes
251–260 of 319 posts
Re: You might not need Kubernetes
#252Earlier quoted context omitted.
> I'm pretty sure the reason they ended up rejecting me was that I mentioned the technical interviewer that I think containers, container infrastructures (like Kubernetes) and even cloud infrastructure is being overused/used without giving too much thought about it as if it came free (in the sense of setup and operating complexity). I'd like to understand your thoughts more on why you believe cloud infrastructure is…
I may have been sloppy to use the cloud infrastructure expression. It can mean different things to different people. Sure thing, you want to use at least VPS-s, from the very beginning. Maybe even blob storage as/if needed. But probably that's it at the beginning. > Put another way: in what way have you seen containers being abused? Well, I wouldn't say abused, but they aren't really necessary a lot of time. E.g. if…
> ... if we talk about a small company with a single product (which, the developers being sane, is a 'monolith' as opposed to being broken into microservices) then containerizing the thing really adds nothing.
I totally agree with the monolith thing. Did that my self recently too. I don't agree about it not being helpful to containerise them, though.
I've containerised a recent Go application I wrote. All it contains is the binary, an assets/ directory for CSS, and a templates/ directory for the HTML server-side templates. Deploying a new version consists of 'make docker-push' followed by 'make docker-run' on the remote server(s). All the Makefile is wrapping is Docker build, push, pull, rm, and run commands. Hardly taxing work.
It's also much easier to version control components when they're all placed into a neat and tidy "box" (image.) And because changes are just diffs of the filesystem, it's absolutely fine having 5,000 versions of your application because like branching in Git, it's cheap.
An example of this is wanting to change a library your application is using and trying it out. Super easy: change the lib', package it into an image with a specific version for that "product", deploy it, it fails (let's say), and you redeploy the previous image and you're back up.
Not to sound too arrogant or obnoxious, but anything other than the above is creating more work for oneself for no reason at all.
> So we use VPSs. Which kind of solves the issue in itself.
Do you pay per VPS? Sounds expensive. Would it not be cheaper and easier to have a "cluster" of VPS and run Docker on them, containerising your application? No need for K8s or any kind of orchestration, but using VMs as a "container" (like using AMIs for example) seems like a heavy, expensive option? I could be wrong.
I think you're solving the problems containers do using a heavier, weightier solution that's also slower and more expensive. Again could be wrong (and happy to be proven so)
> A kernel exploit is enough to break out of them
It would have to be an exploit that gets you a shell or the ability to execute code on the host. They're quite rare, I believe.
Re: You might not need Kubernetes
#253Earlier quoted context omitted.
I once got an interview from a company in the container space because one of their exec read an article I published talking about the trouble with container systems[1]. (Really good talk/interview, but I ended up not moving forward because I didn't want to move back to the west coast). I've been in smaller shops that wasted a lot of time on K8s stuff and fell behind on their timeline. If you want to run k8s, DC/OS, e…
My friend is trying his luck with his own start up, they have yet to launch a 1.0 of product but the CTO implemented K8s citing scalability. I legit laughed at that statement.
My friend has a startup and a single person got a k8s cluster running on aws with kops+GPUs in a couple weeks. He loves it. The people who are running it successfully don't come on here to complain.
Re: You might not need Kubernetes
#254Some day I would like a powwow with all you hackers about whether 99% of apps need more than a $5 droplet from Digital Ocean, set up the old-fashioned way, LAMP --- though feel free to switch out the letters: BSD instead of Linux, Nginx instead of Apache, PostgreSQL instead of MySQL, Ruby or Python instead of PHP. I manage dozens of apps for thousands of users. The apps are all on one server, its load average around…
They do not, I run tens of low-traffic projects very successfully on a $10/mo Hetzner server on Dokku. Dokku is amazing and so is Hetzner, I don't know why people always go for the high-scalability, expensive options just to end up with 0 utilization.
Because my company is risk-averse, and perfectly happy to drop $1000 a month for a ridiculously overprovisioned database instance just to ensure an issue with the database will never cause their contracts to be lost.
Re: You might not need Kubernetes
#255Earlier quoted context omitted.
Kubernetes isn't the only thing around. Kubernetes and Mesos are kinda the heavyweight solutions, but there are smaller things around like Hashicorps Nomad and Swarm, and probably a lot more I don't know. We're currently evaluating nomad, and it's surprisingly pleasant. Nomad doesn't solve every problem every application in every situation might have. Nomad schedules containers, VMs or whatever else on hosts. This re…
Just came here to second this. We evaluated both Kubernetes as well as Nomad for a relatively small cluster of some worker nodes and web services. In the end, the ease of standing up a Nomad cluster and the whole feel of the thing won us over. Nomad is a single golang binary that you can run on your laptop and have a fully working working Nomad client and server, along with a builtin UI and command line tools (same b…
Kubernetes is a single binary, too (hyperkube).
Re: You might not need Kubernetes
#256Earlier quoted context omitted.
Can you give an idea how big your cluster is? 10, 10s, 100s? I'm curious how well Normand works in reality, but it's hard to find ppl running it.
Right now we're 4 weeks in and IT and 2 guys want it. We're 3 server nodes + 2 client nodes big, 128G RAM, 16 cores, 32 threads. Please go ahead and call this tiny and irrelevant. I'm aware and under the same assumption. I've dealt with individual servers around 4x that size. But, go ahead and ask me again in 3 or 6 month. We'll migrate around 10 HW servers as nomad clients, we'll probably get HW capacity from our si…
The problem is if you come on here and talk about how great nomad is, then only later say you are only running five nodes, that gives people false impressions. Kubernetes is tested regularly with 5000 nodes. To get to that level takes an entirely different level of software to be able to run a cluster that large reliably without issues.
Re: You might not need Kubernetes
#257You probably don't need microservices either - it's insane how much money and time is being thrown away to these industrial strength hammers by companies that simply don't need it.
So true! I think the Rick & Morty reference alone speaks volumes for everything. haha
Re: You might not need Kubernetes
#258My experience is that Kubernetes is too complex for the average functioning product. At our company, everyone is obsessed with it because it promises no cloud vendor lock ins! But at what cost? The complexity. Also the direction cloud vendors are going in my opinion, is more hardware-centric (eg. TPUs). How will you avoid the cloud lock in when only Azure offers the image tagging machine learning as a service? How wi…
I'm not a k8s expert, but I'm pretty sure it has an extensible architecture (plugin or similar), so it probably would allow for the definition of an image tagging service interface that could be satisfied by Azure or whomever. Would love to hear from someone more knowledgable than I.
Re: You might not need Kubernetes
#259What bothers me about k8s is that it promises a lot ("15 years of experience of running production workloads at Google" at your fingertips! yay!) but it's in fact still a young, ever-changing solution. Even developing an app locally with minikube is a PITA for a lot of reasons. From Helm to Telepresence to Skaffold, every tool out there is just unpolished and overambitious. Don't want to imagine how those problems mi…
Skaffold is only 5 months old. That's a little unfair to call it unpolished an over-ambitious.
Re: You might not need Kubernetes
#260Earlier quoted context omitted.
There's a bit more to the story than simply solving a problem. I have other plans for the software and we have ideas about how we want to move it forward. One of the key features is being able to "tag" an item on someone's wish list as "bought" or "buying". This allows others viewing the list to know that item has been taken. But there's also a requirement that the original author of the list/item cannot see that it…
Still, I highly doubt that there are not any apps that do this already. I mean, my family uses Amazon wish lists and set it so that you can't see when stuff is purchased.
Like I said, it goes beyond just solving the problem. It's a learning exercise as well as a solution.
Have you never wondered what projects you can write to help you learn X or Y?