Live data from Hacker News

I am building a cloud

crawshaw.io

551–560 of 589 posts

Re: I am building a cloud

#551
post #245

Earlier quoted context omitted.

It was a weird point to make in the post given that exe.dev charges $0.07/GB for transfer. That's arguably worse than the major clouds, who charge about the same for egress but give you free ingress.

Author here. I need to fix our transfer pricing. (In fact I'm going to go look at it now.) I set that number when we launched in December, and we were still considering building on top of AWS, so we put a conservative limit based on what wouldn't break the bank on AWS. Now that we are doing our own thing, we can be far more reasonable.

Love the attitude!! Well done, and good luck with all this. I sent you an email offering help, if any is needed/welcome.

Re: I am building a cloud

#552

AWS. Months of complex dev work to build using their CDK. Terrible disk speed. Frustrating permissions systems. Tiny deployments that take 30 minutes. Rollbacks that get stuck for hours. What you end up with is about 4 CPUs and 16Gb of RAM for $1000+ per month. No wonder Bezos could send his wife and Katie Perry on a jolly into space. The world's richest man 1 IOP at a time. For that money I can get 5 big bare metal…

If you're using cloudflare tunnels, you don't even need to be on OVH. You could seriously host anywhere, like your own basement.

I have a basement, and a garage, and 2 sheds... I still dont wanna run servers at home. I use OVH, I just rent a server at their Hillsboro OR, US datacenter, just a few miles from my home, i know the twists and turns the fiber goes between there even.

Re: I am building a cloud

#553

Earlier quoted context omitted.

I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup. Point being, it's not the tools the causes the probem.

Just curious, are you still looking for developers? Asking as someone who is a developer that works with POS systems.

I no longer work in that industry.

Re: I am building a cloud

#554

Earlier quoted context omitted.

I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup. Point being, it's not the tools the causes the probem.

Read this as a Piece of sh... company. Then I saw response of someone saying they're a POS developer and was like oh I think he means point of sale. Or that guy is just a really bad programmer.

Both work (:

But the point was it was in a comparble situations without the microservices / k8s / whatever pet tech you want to hate on.

Re: I am building a cloud

#555

Earlier quoted context omitted.

I ran renderapp in ECS before I ran it in k8s. The deployment files / structure were mostly equivalent with the main differences being I can't shell into ECS and I lose kubectl in favour of looking at the AWS GUI ( which for me is a loss, for others maybe not ). The main difference is k8s has a lot of optionality, and folks get analysis paralysis with all the potential there. You quickly hit this in k8s when you have…

> I can't shell into ECS Is there a specific reason why you can't shell into ECS? IIRC, I was able to do so by following the guide [0]. [0] https://aws.amazon.com/blogs/containers/new-using-amazon-ecs...

No, I was simply wrong. Thanks for pointing that out.

Re: I am building a cloud

#556

> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…

If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

[deleted]

Re: I am building a cloud

#557
post #311

Earlier quoted context omitted.

People have it backwards. If you have an app and you want to run a single app yeah silly to look for K8s. If you have a beefy server or two you want to utilize fully and put as many apps on it without clashing dependencies you want to use K8s or docker or other containers. Where K8s enables you to go further.

Why would you want to use K8s for one or two beefy servers? It's designed for solving a different problem at a large scale.

I think automatic scaling is useful to utilize server fully - apps that don't need resources automatically scale down, apps that need resources can auto scale up.

I bet you can do it in some other way but that's built in feature of k8s.

Re: I am building a cloud

#558
I use kubernetes extensively at work. I don't manage the kubernetes cluster anymore since now we have a team that runs centralized services and you can request a namespace with a quota. But back when my team had a dedicated Azure Kubernetes cluster it was not that bad as people says it is and the biggest hassle was the extremely short lived support for each version.

Then I started to realize most people who complain are rolling their own which is also not bad since there are products like k3s that are very simple to use.

It seems things start to fall apart when they try to stuff it with all kinds of crazy idiotic controllers and the favorite of the month CNI and CSI. I always shake my head when I see people creating sand castles by setting up stuff like Ceph from within the cluster.

If you want to play with it keep things simple and have all the persistent data outside of the cluster. Use good old NFS instead of the latest longceph horngluster version. Keep databases and the container registry out. Treat it like a compute pool not a virtual datacenter. Stop recursing chickens inside eggs.

Re: I am building a cloud

#559

Earlier quoted context omitted.

Docker is great development tooling (still some rough edges, of course). Docker Compose is good for running things on a single server as well. Docker Swarm and Hashicorp Nomad are good for multi-server setups. Kubernetes is... enterprise and I guess there's a scale where it makes sense. K3s and similar sort of fill the gap, but I guess it's a matter of what you know and prefer at that point. Throw on Portainer on a s…

> Docker is great development tooling (still some rough edges, of course). Show me a Docker in use where build caching was solved optimally for development builds (like eg. make did for C 40 or 50 years ago)? Perhaps you consider Docker layers one of the "rough edges", but I believe instant, iterative development builds are a minimum required for "great development tooling". I did have great fun optimizing Docker bui…

A multi-stage Docker build where you separate pulling in dependencies from building the thing you want is as close as you're going to get.

Something like the following works well in practice:

  1) pinned base image (e.g. Ubuntu LTS)
  2) your own custom base image in a registry rebuilt whenever you want (e.g. with tools you need for debugging or available across all of your images)
  3) your own runtime-specific base image, like a JDK one, can be used later both as a basis for development images with additional tooling, as well as for runtime images of your app
  4) your own runtime-specific development images, like one that's based on the JDK image above + Maven, alongside any other development tooling you need
  5) your multi-stage application image, where the first stage uses the development image to COPY in the dependency description files you need and then pull the dependencies, then does the build (layer cache takes care of reusing things where possible), and then the second stage is based on the runtime image (e.g. JDK) where you just copy your finished artifact (e.g. .jar file)
If you don't need or want to build your own images, you can fold steps 1-4 into just using upstream images off of Docker Hub or whatever you prefer, but in practice it works pretty okay across numerous stacks. Of course, it's also possible to easily have very high standards in regards to what you mean as "optimal", so Docker probably won't live up to that.

Re: I am building a cloud

#560
all of the grievances resulting in this move is a simple outcome of the cost of convenience. but it should not need going full opposite end to get something good enough.

dedicated servers, as hinted by others here, addresses the vast majority of issues one may face for any non-enterprise needs. if you know about IOPS and care about them, odds are that running a simple open-source project [1] on top of one is all you need to do to move on with your day.

need redundancy, etc.? can complement with another one in another provider/region or put CF in front of your box. this is clearly working well enough for some of the commenters who are able to sell their own service on top of this approach.

[1] https://disco.cloud/

Post reply on HN