Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rolling your own servers with Kubernetes

gravitational.com

Post reply on HN
Print view View original

201–209 of 209 posts

Re: Rolling your own servers with Kubernetes

#201

Post by neop1x »

gambler wrote: post #53

>Should you roll your own servers? If you are not certain, the answer is most likely “no”. The staggering growth of AWS happened for a reason. Funny how for many decades companies and people were running their own servers. The hardware was getting cheaper each year. More software became available via open source. Then several ubecorporations entered the hosting/cloud business, and suddenly no one seems to be able to…

Here is an awesome presentation from Rancher about how cloud providers are not providing resources but convenience and want people to buy more and more services. https://m.youtube.com/watch?v=mnu477057W0 full slides etc are here after registration if interested https://info.rancher.com/single-node-standalone-servers-scal...

Re: Rolling your own servers with Kubernetes

#202

Post by neop1x »

acd wrote: post #42

There is also kubespray Ansible provisioning of Kubernetes, kops and kubeadm. Rolling Kubernetes on your own is quite hard though especially the networking part so there should be a market for a company helping out with provisioning. Using public cloud can be very expensive for some work loads which were not written for the cloud due to higher resource usage. It is still quite hard to forecast cost of the cloud due t…

It is better now with kubeadm. There are still lots of things to learn and lots of breaks which can happen but after months of learning I now have my hown cluster and it works perfectly. It mounts NFS store, also use local storage and I plan to play with OpenEBS. Nginx ingress + certbot (let's encrypt auto-renowal) works fine. Logging can be done with Fluentd and Graylog. Prometheus for performance measures. MetalLB as load-balancing replacement on L2. Calico for networking and networkPolicies. Cloud providers definitely have some added benefits, less friction. But 'do-it-yourself' is well possible as wrll and it helps you to understand more individual pieces K8S is made of and how to debug those.

Re: Rolling your own servers with Kubernetes

#203

Post by apple4ever »

ranrotx wrote: post #151

AWS employee here--thoughts and opinions are my own. Prior to AWS, I was in IT Operations at a large financial services company. I saw the writing on the wall that over time, companies would not want to manage this part of their IT infrastructure themselves. Keep in mind, I was someone who was responsible for keeping the lights on for a decent number of Linux severs. For an individual company, there really isn't much…

What is missing from this discussion is level of control and tradeoffs.

First, having a full level of control can be desirable.

Second, you trade off maintaining firmware levels on hardware and software levels on hardware to managing the way cloud providers build networking and servers themselves (and the cost control as well).

Maintaining your own hardware and keeping it up to date really isn't as bad as people make it out to be. Its not hard to get right either, assuming you can hire any sort of decent system engineer.

Re: Rolling your own servers with Kubernetes

#204

Post by apple4ever »

marcinzm wrote: post #165
gambler wrote: post #91

Earlier quoted context omitted.

>Most companies are not like Facebook or Dropbox that can maintain an internal IT organization at a high level equivalent to AWS. Let's try this with different phrasing. In 200X a lot of companies were able to maintain their own infrastructure, just like Facebook and Amazon did at the time. Forward 10-13 years. We have cheaper hardware. We have extra 10+ years of development in open-source software. And yet that list…

>In 200X a lot of companies were able to maintain their own infrastructure, just like Facebook and Amazon did at the time. I worked at a decently large tech-ish company at that time. It took 3 months to provision a handful of dev servers in the data center. After filling out forms in triplicate. We asked for extras since we had almost zero ability to manage them (reboot, etc.) or get them re-built if something broke.…

That sounds like a problem with process, not technology.

I can provision a handful of production servers in 5 minutes.

Re: Rolling your own servers with Kubernetes

#205

Post by apple4ever »

lixtra wrote: post #95
gambler wrote: post #91

Earlier quoted context omitted.

>Most companies are not like Facebook or Dropbox that can maintain an internal IT organization at a high level equivalent to AWS. Let's try this with different phrasing. In 200X a lot of companies were able to maintain their own infrastructure, just like Facebook and Amazon did at the time. Forward 10-13 years. We have cheaper hardware. We have extra 10+ years of development in open-source software. And yet that list…

Ok, I will try to counter that. The stack grows higher. More specialized employees become more effective. In 200X you just needed a Linux admin. Now on top of being good a good Linux admin you also need to know docker, kubernetes (the admin part, not the user part) etc. If you don’t need that daily, because you also need to maintain some apps, you will be less professional than a cloud provider. Of course it’s a trad…

Really good points. Maintaining your own infrastructure may be "boring" with "old" technologies. But its a known process. Its really easy.

But with the cloud its a whole different domain. It may be easy too, but I don't think its as easy as hosting your own.

And your last two lines are great. It really comes down to the company about which tradeoff is appropriate.

Re: Rolling your own servers with Kubernetes

#206

Post by apple4ever »

sroussey wrote: post #162
hn_throwaway_99 wrote: post #107

Earlier quoted context omitted.

I worked at a couple of those self-hosted companies in the 200Xs. For me, as a developer, it was painful and way, way, way worse than AWS or GCP. Planning for big expected spikes of traffic was extremely painful, and once the cloud vendors started adding new services we felt continually behind the times. I can only think it was "interesting" if you've forgotten how difficult and expensive managing your own infrastruc…

Spiky usage patterns are great for cloud. Otherwise, I’ve did some tests years ago and what I spent over 5 years would have gotten me only a few months on AWS. And I think I only walked into the data center once in those five years.

When I did tests and studies for our new infrastructure, AWS was at least 2x as expensive per year. And in the 4-5 year range, it made it 3x+ expensive.

Re: Rolling your own servers with Kubernetes

#209

Post by compuguy »

numlock86 wrote: post #208
prisar wrote: post #207

link is broken

Yup, 404. Even manually going to their blog I can't find the entry anymore. Bookmarked because it was supposed to be an ongoing series with multiple parts. Weird provider. Probably best to avoid.

I have a feeling they deleted the blog post. There is an archived copy here: https://web.archive.org/web/20190508123542/https://gravitati...
Post reply on HN