And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Docker operations slowing down on AWS
51–60 of 176 posts
Re: Docker operations slowing down on AWS
#52And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Sadly their support quality has decreased and our own account team has ignored us for weeks so maybe stick with AWS instead.
Re: Docker operations slowing down on AWS
#53And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
Ah, that nasty AWS vendor lockin. Guess I'll just run my own CDN then, shouldn't be hard (can't use other vendors, because lock-in). L'il ol me running edge nodes all over the five continents where we have clients. I'll have it done by closing time today. AWS is more than just compute cores, and you're hand-waving away what it offers.
If you're talking about actually running a global application then that would be a scenario where a network like Google's or Softlayer's does help by having 1 giant VLAN.
Re: Docker operations slowing down on AWS
#54Earlier quoted context omitted.
Yes. Look at https://www.packet.net/bare-metal/ for bare metal boxes provisioned in less time than an ec2 instance. Or if you want to go old school, cheaper, and less sexy/api driven: https://www.delimiter.com/ If you shop, for around 30-40 a month you can get 16 cores, 32gb ram, and a 120GB ssd or 1-2TB spinny disk.
also scaleway.com (they have API, but no user-data). IMO, docker and container orchestration spells a bright future for bare-metal boxes like these, as you won't need cloudformation, etc.. But I still see few alternatives to S3, many vendors offers block devices, but only the big clouds offer blob storage. Backup and restore from blob storage makes recovery from crash pretty easy.
Re: Docker operations slowing down on AWS
#55And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
> "My take: a single machine (or two for HA) will be enough"
2 bare metal instances isn't HA. Not even close.
> "if you really want to go big separate the web server from the database but that's it."
I would always recommend separating the web server from the database server on anything professional. It gives an easy clear path for scaling sideways (since you've already separated out your back end from your application), it allows you to tighten security (eg only allow access to the DB server from the web servers via the unprivileged DB user connection), it also makes maintenance easier. Even if you're only running on one physical box, put the web server and DB in their own VM or LXC/Zone/Jail container.
> The other day I saw quad E7-4870 (yeah won't win any single thread contest but has 40 cores and 80 threads) 512GB RAM servers for $299 a month, with 1TB RAM for $499. Had a low end 2TB SSD for boot and you could add 8x1TB HDD w/ HW RAID for $40...
I work with both bare metal servers matching your description and both self-hosted and private clouds. Frankly I think your rant misses one of the most important point of working with AWS and that's the convenience and redundancy that the tooling offers. AWS isn't just about single instances, it's about having redundant availability zones with redundant networking hardware and about being able to have disaster recovery zones in whole other data centres and having all of the above work automatically. Getting our self hosted stuff to even close the level of tooling that AWS offers took months of man hours and quite a considerable more initial set up costs. Having to buy at least two of every piece of kit for redundancy, having to have BT lay two dedicated internet links (we have 3 now) just incase a builder accidentally cuts one of our lines and having core infrastructure replicated off site all adds considerably to both the set up time and cost. So yeah, for small businesses and personal blogs AWS is a bit overkill. But you cannot use the "high availability" argument and say "2 physical machines is enough".
Disclaimer: I've worked for clients such as Sony, UEFA and News International as well as many smaller but still sizable national publications. Our infrastructure has consisted of both scaled up physical hardware and scaled sideways virtual machines and frankly I/we wouldn't be able to offer the kinds of services we do nor with the kind of uptime we do without running a fleet of virtualized web servers.
Re: Docker operations slowing down on AWS
#56And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
We would be investing far more time into system operations than I do now thanks to AWS's automation of standard stuff. My hosting bill would increase - partly because we can run on t2.micro instances, but that awfully glib advice about overprovisioning is most definitely asking for trouble; and we'd lose curated services like RDS and OpsWorks - which, by the way, are PostgreSQL and Chef, hardly a "ugly vendor lock" but simply well-designed infrastructure services based on standard parts. Oh, and I'd have to spend more money and time on auditors, because bare-metal providers don't have compliance programmes like AWS/Azure/GCP do. And I've haven't even started to think about securing the resulting systems to the same level I get for minimal effort from a major public cloud.
You're not a dinosaur, no, but these claims don't match my reality, nor that of many other projects besides. This is compounded in my view because you've claimed specific expertise to promote your opinion. Any manager bringing me such a poorly considered and bombastically argued business case would be sent away with an ear bashing about TCO and opportunity cost.
Re: Docker operations slowing down on AWS
#57And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
If I followed this advice, it would hurt my startup badly. We would be investing far more time into system operations than I do now thanks to AWS's automation of standard stuff. My hosting bill would increase - partly because we can run on t2.micro instances, but that awfully glib advice about overprovisioning is most definitely asking for trouble; and we'd lose curated services like RDS and OpsWorks - which, by the…
Some startups (A) I work with have basically no OPS costs beyond setup, integrating Docker deployements and getting automatic backup working. Most simple technology just works and devs easily can do operations. The largest pain point still is VPN. Machines today are very very fast and load of many startups is very low. This are mostly simple marketplace startups etc. without any rocket science aka "web ui frontend to database". They often have Some startups (B) I work with have high OPS costs due to demanding technology needs for throughput, load peaks, amount of data with innovative technology at their core, aka "rocket science".
I have not seen any correlation between AWS usage and A/B types.
From my experience with startups the only way to successfully use AWS is deep integration and using lots of services. If you use AWS and do everything on your own people are doing it wrong.
Re: Docker operations slowing down on AWS
#58And then people consider me a dinosaur when I say, no cloud, just rent a server or two (not colo! just dedicated servers). Your average web service does not need to scale near infinitely; for the same amount of money you pay to Amazon you can overprovision 3-5-10 times and that'll handle your spikes. No surprises. Same amount of work: EC2 and bare metal both gives you a root prompt, go from there. These days you can…
I don't disagree with your point per se as I do love working with physical gear, but I do think you're grossly missing the point in places: > "My take: a single machine (or two for HA) will be enough" 2 bare metal instances isn't HA. Not even close. > "if you really want to go big separate the web server from the database but that's it." I would always recommend separating the web server from the database server on a…
Even then the answer is: it depends what you need. I run blogs on S3 + CloudFront. That's effectively content versioning and geo-distributed caching CDN for pennies. AWS is not just EC2.
Re: Docker operations slowing down on AWS
#59Earlier quoted context omitted.
The problem with renting boxes is the hidden costs if you want to do it right. First of all, if you have anything mission critical, you need to run it in a high availability config, this is easy for stateless microservices, but when it comes to running your DB, you start renting three boxes instead of one or two and configuring them accordingly. And then you setup your Backup Infrastructure for disaster recovery, Gla…
I completely agree with your line of reasoning and I have a few more thoughts on the matter. There are a ton of "soft" problems that arise when building out your stack. Licensing, warranties, and maintenance issues are things that never come up with a cloud provider. Then there are physical problems with building your own cabinet like cabling and power & heat management. The list just grows and grows. Another big thi…
Did I stutter? I said no colo. You need to be ridiculously large for colo to be worth considering.
Re: Docker operations slowing down on AWS
#60Earlier quoted context omitted.
I don't disagree with your point per se as I do love working with physical gear, but I do think you're grossly missing the point in places: > "My take: a single machine (or two for HA) will be enough" 2 bare metal instances isn't HA. Not even close. > "if you really want to go big separate the web server from the database but that's it." I would always recommend separating the web server from the database server on a…
> So yeah, for small businesses and personal blogs AWS is a bit overkill. Even then the answer is: it depends what you need. I run blogs on S3 + CloudFront. That's effectively content versioning and geo-distributed caching CDN for pennies. AWS is not just EC2.