Earlier quoted context omitted.
It was a weird point to make in the post given that exe.dev charges $0.07/GB for transfer. That's arguably worse than the major clouds, who charge about the same for egress but give you free ingress.
Author here. I need to fix our transfer pricing. (In fact I'm going to go look at it now.) I set that number when we launched in December, and we were still considering building on top of AWS, so we put a conservative limit based on what wouldn't break the bank on AWS. Now that we are doing our own thing, we can be far more reasonable.
I am building a cloud
551–560 of 589 posts
Re: I am building a cloud
#552AWS. Months of complex dev work to build using their CDK. Terrible disk speed. Frustrating permissions systems. Tiny deployments that take 30 minutes. Rollbacks that get stuck for hours. What you end up with is about 4 CPUs and 16Gb of RAM for $1000+ per month. No wonder Bezos could send his wife and Katie Perry on a jolly into space. The world's richest man 1 IOP at a time. For that money I can get 5 big bare metal…
If you're using cloudflare tunnels, you don't even need to be on OVH. You could seriously host anywhere, like your own basement.
Re: I am building a cloud
#553Earlier quoted context omitted.
I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup. Point being, it's not the tools the causes the probem.
Just curious, are you still looking for developers? Asking as someone who is a developer that works with POS systems.
Re: I am building a cloud
#554Earlier quoted context omitted.
I took over tech for a POS company some years ago. They were a .net shop with about 80 developers, less than 200 concurrent connections, 6 figures spend cloud, and 0 nines uptime with a super traditional setup. Point being, it's not the tools the causes the probem.
Read this as a Piece of sh... company. Then I saw response of someone saying they're a POS developer and was like oh I think he means point of sale. Or that guy is just a really bad programmer.
But the point was it was in a comparble situations without the microservices / k8s / whatever pet tech you want to hate on.
Re: I am building a cloud
#555Earlier quoted context omitted.
I ran renderapp in ECS before I ran it in k8s. The deployment files / structure were mostly equivalent with the main differences being I can't shell into ECS and I lose kubectl in favour of looking at the AWS GUI ( which for me is a loss, for others maybe not ). The main difference is k8s has a lot of optionality, and folks get analysis paralysis with all the potential there. You quickly hit this in k8s when you have…
> I can't shell into ECS Is there a specific reason why you can't shell into ECS? IIRC, I was able to do so by following the guide [0]. [0] https://aws.amazon.com/blogs/containers/new-using-amazon-ecs...
Re: I am building a cloud
#556> Making Kubernetes good is inherently impossible, a project in putting (admittedly high quality) lipstick on a pig. So well put, my good sir, this describes exactly my feelings with k8s. It always starts off all good with just managing a couple of containers to run your web app. Then before you know it, the devops folks have decided that they need to put a gazillion other services and an entire software-defined netw…
If you spin up Kubernetes for "a couple of containers to run your web app", I think you're doing something wrong in the first place, also coupled with your comment about adding SDN to Kubernetes. People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.
Re: I am building a cloud
#557Earlier quoted context omitted.
People have it backwards. If you have an app and you want to run a single app yeah silly to look for K8s. If you have a beefy server or two you want to utilize fully and put as many apps on it without clashing dependencies you want to use K8s or docker or other containers. Where K8s enables you to go further.
Why would you want to use K8s for one or two beefy servers? It's designed for solving a different problem at a large scale.
I bet you can do it in some other way but that's built in feature of k8s.
Re: I am building a cloud
#558Then I started to realize most people who complain are rolling their own which is also not bad since there are products like k3s that are very simple to use.
It seems things start to fall apart when they try to stuff it with all kinds of crazy idiotic controllers and the favorite of the month CNI and CSI. I always shake my head when I see people creating sand castles by setting up stuff like Ceph from within the cluster.
If you want to play with it keep things simple and have all the persistent data outside of the cluster. Use good old NFS instead of the latest longceph horngluster version. Keep databases and the container registry out. Treat it like a compute pool not a virtual datacenter. Stop recursing chickens inside eggs.
Re: I am building a cloud
#559Earlier quoted context omitted.
Docker is great development tooling (still some rough edges, of course). Docker Compose is good for running things on a single server as well. Docker Swarm and Hashicorp Nomad are good for multi-server setups. Kubernetes is... enterprise and I guess there's a scale where it makes sense. K3s and similar sort of fill the gap, but I guess it's a matter of what you know and prefer at that point. Throw on Portainer on a s…
> Docker is great development tooling (still some rough edges, of course). Show me a Docker in use where build caching was solved optimally for development builds (like eg. make did for C 40 or 50 years ago)? Perhaps you consider Docker layers one of the "rough edges", but I believe instant, iterative development builds are a minimum required for "great development tooling". I did have great fun optimizing Docker bui…
Something like the following works well in practice:
1) pinned base image (e.g. Ubuntu LTS)
2) your own custom base image in a registry rebuilt whenever you want (e.g. with tools you need for debugging or available across all of your images)
3) your own runtime-specific base image, like a JDK one, can be used later both as a basis for development images with additional tooling, as well as for runtime images of your app
4) your own runtime-specific development images, like one that's based on the JDK image above + Maven, alongside any other development tooling you need
5) your multi-stage application image, where the first stage uses the development image to COPY in the dependency description files you need and then pull the dependencies, then does the build (layer cache takes care of reusing things where possible), and then the second stage is based on the runtime image (e.g. JDK) where you just copy your finished artifact (e.g. .jar file)
If you don't need or want to build your own images, you can fold steps 1-4 into just using upstream images off of Docker Hub or whatever you prefer, but in practice it works pretty okay across numerous stacks. Of course, it's also possible to easily have very high standards in regards to what you mean as "optimal", so Docker probably won't live up to that.Re: I am building a cloud
#560dedicated servers, as hinted by others here, addresses the vast majority of issues one may face for any non-enterprise needs. if you know about IOPS and care about them, odds are that running a simple open-source project [1] on top of one is all you need to do to move on with your day.
need redundancy, etc.? can complement with another one in another provider/region or put CF in front of your box. this is clearly working well enough for some of the commenters who are able to sell their own service on top of this approach.