Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New setup for 2020

changelog.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: New setup for 2020

#41

Post by shaicoleman »

brobinson wrote: post #28
PietKachelhout wrote: post #26

I'm disappointed that blogs and podcasts keep promoting Linode. I currently maintain about 30 Linodes and I have been doing so for the past 2 years. Some things I noticed: * The internal network is not private. But people don't realise it. You share a /16 with other Linodes. So many open databases, file shares and other services in there. * Block storage performance is really poor, around 100 iops. Same as a SATA dis…

With all these negatives, there must be a really compelling reason to stay. What is it?

In a previous job a couple of years ago, I experienced regular issues with Linode (hypervisor errors, storage errors, performance issues, networking issues, etc.).

Despite all that, management decided to stay with Linode for the following reasons:

* Change is hard, and "better the devil you know" mentality.

* The instance pricing looks cheap compared to AWS. e.g. c5.xlarge ($124) vs 8GB-Balanced ($40) that Linode charges. In reality it isn't so cheap because it's poor oversold technology.

* AWS/GCP have exorbitant bandwidth pricing. Linode bandwidth is very generous, as its pooled across all servers in the account.

* Having someone to pick up the phone 24/7 when there's a problem is a big plus in theory. However, it's much better not to need to call in the first place because things just work.

* Migrating providers can be an expensive and time-consuming endeavour.

* Technical debt, interdependencies, manually configured snowflake servers and infrastructure, no documentation, etc. makes changes risky.

* Not enough DevOps on the team, and too many fires to put out, and shiny features to ship means cloud provider migration is low on the priority list.

Re: New setup for 2020

#42

Post by ithkuil »

lifeisstillgood wrote: post #16

>>> We no longer provision load balancers, or configure DNS; we simply describe the resources that we need, and Kubernetes makes it happen. This is (part) of what keeps me in the stone age. You are provisioning load balancers and DNS - but just one step removed through k8s And my prior is that we need to understand that, be aaare of it, have a model of what is going on to help develop and debug. And so it feels a bit…

Let's turn the knobs on the scenario and see how it appears:

>>> >>> with the advent of the first programming languages you no longer had to think in terms of registers, loading operands from memory, storing the results back, spilling registers to the stack.

>>> You _are_ handling registers and memory spills, but just one step removed through the use of C.

The analogy may not be perfect, but I think it makes obvious some of the things also mentioned in sibling comments: it's all about habit, maturity and thus trust.

If you trust your tools are working correctly, if you know how to deal with their well known quirks, you'll just rebase on top a layer and hopefully boost your productivity and tackle more complex problems more easily.

Maturity is important because if today you're more likely to blame yourself than the compiler when your stuff doesn't works, it's just because you're lucky to work with mature and popular toolchains. (And I'm not only talking about the past when compilers where new and unproven, it still happens today on some niche embedded toolchains).

So, yes, it's indeed rational to be wary of new unproven abstraction layers as they could bring more pain than help.

It's hard to judge when that line is crossed though.

I personally like to know how stuff works under the hood anyways. I find it useful in practice and it gives me confidence in using the higher layers, when they make sense, or stay with the lower level layers, when they make sense.

Occasionally I still write some assembly. But most of the time, for most of the stuff, it just makes more sense to use a higher level programming language.

I see k8s in a similar way. We have operating systems, programming languages, etc; all sorts of abstractions that help us separate concerns and have specialists dealing with the nitty gritty details of some stuff, so that everybody else can be specialized in something else (just like in real life)

Re: New setup for 2020

#43

Post by threwawaysoff »

PietKachelhout wrote: post #26

I'm disappointed that blogs and podcasts keep promoting Linode. I currently maintain about 30 Linodes and I have been doing so for the past 2 years. Some things I noticed: * The internal network is not private. But people don't realise it. You share a /16 with other Linodes. So many open databases, file shares and other services in there. * Block storage performance is really poor, around 100 iops. Same as a SATA dis…

I ditched Linode years ago because it just wasn't that great. AWS if I need ephemeral boxes (I used to be an enterprise on-prem AWS consultant), but I run most everything on a home 96 EPYC core, 512 GB, SSD, HDD (ZFS) box running KVM, Docker, and open vswitch. It just isn't worth it to rent slow, expensive servers when I need lots of them and to be fast. I don't have any problems remoting into them with ddns and wireguard.

Re: New setup for 2020

#44

Post by ClumsyPilot »

gerhardlazu wrote: post #38
lifeisstillgood wrote: post #16

>>> We no longer provision load balancers, or configure DNS; we simply describe the resources that we need, and Kubernetes makes it happen. This is (part) of what keeps me in the stone age. You are provisioning load balancers and DNS - but just one step removed through k8s And my prior is that we need to understand that, be aaare of it, have a model of what is going on to help develop and debug. And so it feels a bit…

K8S is an API that the majority is agreeing on, which is rare. There is a lot of amazing tooling, a staggering amount of ongoing innovation, all built on solid concepts: declarative models, emitted metrics (the /proc equivalent, but with larger scope) and versioned infrastructure as data (a.k.a. GitOps). For someone that is known as the King of Bash (self-proclaimed) - https://speakerdeck.com/gerhardlazu/how-to-write…

I thi k you've hit your head on the nail - the point is not just the kubernetes, it's that you can build standard infrastructure on top. Any software can be (in theory) setup with a helm script, configured in a standard way through YAML configmaps rather than some esoteric configfiles or scripts which are diffetent for every piece of software

Re: New setup for 2020

#45

Post by manigandham »

gerhardlazu wrote: post #36
manigandham wrote: post #19

The static files should definitely be on some kind of object storage like S3, that's what it's built for. Much faster, more reliable, more scalable, and likely much cheaper too. As for persistent volumes, might be better to just offload Postgres to a managed DB service and downsize the K8S instances, or use something like CockroachDB which is natively distributed and can make use of local volumes instead.

Yes, it does make sense to move static files to object storage, especially the mp3s. There is some ffmpeg-related refactoring that we need to do before we can do this though, and it's not a quick & easy task, so we have been deferring it since it's not that high priority, and there are simpler solutions to this particular problem (i.e. improved CDN caching). Other static files such as css, js, txt make sense to remai…

Yea the small static files that are part of your webapp can stay with it, but media files are best on S3. If you need a block interface though, I recommend something like ObjectiveFS: https://objectivefs.com/

If you're using an operator then local volumes is a good middleground if it automates the replication already. CockroachDB also has a kubernetes operator although it's only for GKE currently. There are also other options like YugabyteDB which is another cloud-native postgres-compatible DB.

Re: New setup for 2020

#46

Post by SergeAx »

I browsed the site swiftly. Isn't it a bunch of content items, like WordPress site? So, here we are, deploying WordPress in the cloud via Kubernetes, for real, not as a funny meme?

Re: New setup for 2020

#47

Post by ojhughes »

theptip wrote: post #23
lifeisstillgood wrote: post #16

>>> We no longer provision load balancers, or configure DNS; we simply describe the resources that we need, and Kubernetes makes it happen. This is (part) of what keeps me in the stone age. You are provisioning load balancers and DNS - but just one step removed through k8s And my prior is that we need to understand that, be aaare of it, have a model of what is going on to help develop and debug. And so it feels a bit…

In the happy path, your developers no longer need to worry about this stuff. It’s possible for a team to stand up a new service and plumb it through all the way to the external LB just using k8s yaml templates. In the unhappy path, sure, you need someone who knows how to debug networking issues, and in some cases it’s going to be harder to debug because of the layers of indirection. But the total amount of toil is si…

I agree and in my experience with k8s, the unhappy path is a rare experience. Everything just works most of the time

Re: New setup for 2020

#48

Post by tracker1 »

js4ever wrote: post #4

I was baffled by this: "The worst part is that serving the same files from disks local to the VMs is 44x faster than from persistent volumes (267MB/s vs 6MB/s)." Is it a configuration issue on their side or do the LKE volumes are really limited to 6MB/s on linode? How can you be happy with this for production??

It's a relatively common problem... if you need more IO, you may want to try a stripe of 4-8 block storage connections.

Then again, depends on what you're doing.

Re: New setup for 2020

#49

Post by gerhardlazu »

threwawaysoff wrote: post #43
PietKachelhout wrote: post #26

I'm disappointed that blogs and podcasts keep promoting Linode. I currently maintain about 30 Linodes and I have been doing so for the past 2 years. Some things I noticed: * The internal network is not private. But people don't realise it. You share a /16 with other Linodes. So many open databases, file shares and other services in there. * Block storage performance is really poor, around 100 iops. Same as a SATA dis…

I ditched Linode years ago because it just wasn't that great. AWS if I need ephemeral boxes (I used to be an enterprise on-prem AWS consultant), but I run most everything on a home 96 EPYC core, 512 GB, SSD, HDD (ZFS) box running KVM, Docker, and open vswitch. It just isn't worth it to rent slow, expensive servers when I need lots of them and to be fast. I don't have any problems remoting into them with ddns and wire…

My first Supermicro just turned 9 and it's still running strong, with a fresh install of Ubuntu 20.04 & k3s over the holidays. The second Supermicro turned 5, and has been running FreeBSD all this time like a champ. They are both loft guardians.

A bunch of bare metal hosts run on Scaleway / Online, and different VMs & managed services run in Digital Ocean, Linode, AWS & GCP. I sometimes spin the odd bare metal instance on Equinix Metal (former Packet).

A diverse fleet means that there's always something new to learn and try out. A single large host would make me anxious, as no internet provider or power grid is 100% reliable and available. Also, software upgrades sometimes fail, and things get messed up all the time, which is when I find it most efficient to just start from scratch. A single host makes that less convenient.

Every approach has its pros and cons, which is why my main workstation is a 20 Xeon W with 64GB RAM & 1TB NVME : ). Yes, there is a backup workstation which doubles up as a mobile one meaning that it can work without power or hard internet for almost a day. Options are good ; )

Re: New setup for 2020

#50

Post by gerhardlazu »

tracker1 wrote: post #48
js4ever wrote: post #4

I was baffled by this: "The worst part is that serving the same files from disks local to the VMs is 44x faster than from persistent volumes (267MB/s vs 6MB/s)." Is it a configuration issue on their side or do the LKE volumes are really limited to 6MB/s on linode? How can you be happy with this for production??

It's a relatively common problem... if you need more IO, you may want to try a stripe of 4-8 block storage connections. Then again, depends on what you're doing.

For what it's worth, Rook, OpenEBS or Longhorn are worth exploring.
Post reply on HN