Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A million-dollar engineering problem

segment.com

Post reply on HN
Print view View original

31–40 of 265 posts

Re: A million-dollar engineering problem

#31

Post by [deleted] »

user5994461 wrote: post #22
Negitivefrags wrote: post #16

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service i…

You shouldn't be on AWS in the first place if everything you do can fit on a single server. Use the right tool for the job.

[deleted]

Re: A million-dollar engineering problem

#32

Post by [deleted] »

user5994461 wrote: post #22
Negitivefrags wrote: post #16

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service i…

You shouldn't be on AWS in the first place if everything you do can fit on a single server. Use the right tool for the job.

[deleted]

Re: A million-dollar engineering problem

#33

Post by rbranson »

kornish wrote: post #26

The Dynamo incident highlights an important lesson when using consistently hashed distributed data stores: make sure the actual distribution of hash keys mirrors the expected distribution. (though to their credit, someone writing an automated test using a hard-coded key was beyond their control). Incidents like this are generally why rate limits exist, which they don't currently have [0], but perhaps they'll consider…

Thanks for the warm feedback!

We currently have an internal project underway to detect hot keys in our pipeline and collapse back-to-back writes before they're written out to DynamoDB.

It's difficult to apply throttling on these conditions synchronously within the ingestion API (i.e. return a 429 based on too many writes to one key) because of the flexibility of the product: that workload is perfectly acceptable for some downstream partners. It also gives me pause from a reliability perspective. We try to keep our ingestion endpoints as simple as possible to avoid outages, which for our product means data loss.

Re: A million-dollar engineering problem

#34

Post by gravypod »

I've always been set aback by how much AWS servers cost. Maybe I'm just too cheap but you can go to very reputable hosting companies and get things at fractions of the price.

For example, if I need 16GB of memory and 4 cores here are my options:

    * AWS (t2.xlarge) $137.62
    * OVH (EG-16)     $79
    * So You Start/OVH (SYS-IP-1S) $42
    * Kimsufi/OVH (KS-3B) $29
    * Hetzner (EX41) €46.41 [Lowest cost model is 32GB of ram]
    * Joe's Data Center (No model name) $45 [Duel socketed L5420 which is 8 cores]
That's a crazy price difference and, unlike frmo what I understand about AWS, you don't need to pay for bandwith for most of these companies or they have some crazy high limit. IIRC it's common for 20TB to be the standard (FREE) bandwith limit. You're also on real hardware, not a VM.

Unfortunately not all of these are perfect systems. There are issues with them. Some have slower CPUs, some have slower disks, some are in other countries but you can just pick the ones that suit your need for whatever you have to run. You can also use VMs that are far cheaper then these dedicated systems. If your workload is under 50% duty cycle on a CPU and under 4GB of memory you don't need a dedicated server. Buy a VM from one of these companies:

    * RamNode (8GB SVZS) $28 [Fully SSD storage, very fast network]
    * OVH (VPS CLOUD RAM 2) $22.39 [12GB(!) of ram and 2 vCores]
    * Linode (Linode 4GB) $20
These are all VPSs but they will get the job done and are cheap enough that you can build something great on a budget.

This is just from a few minutes of remembering when I had to buy my own hardware. I'm sure this isn't an exhaustive list. You can usually find information on the web hosting talk forums and on a site called low end box. I don't have links on hand but they're worth a read.

Re: A million-dollar engineering problem

#35

Post by rbranson »

dastbe wrote: post #12

Reading through this, I would change "Dynamo is Amazon’s hosted version of Cassandra" to "DynamoDB is Amazon's hosted key-value store, similar to Cassandra". The former (to me) sounds like you're saying they vend a managed Cassandra.

You are totally right. This was an oversight that we didn't catch during our review process. Thanks so much for the tip, we've updated the post.

Re: A million-dollar engineering problem

#37

Post by djchen »

gravypod wrote: post #34

I've always been set aback by how much AWS servers cost. Maybe I'm just too cheap but you can go to very reputable hosting companies and get things at fractions of the price. For example, if I need 16GB of memory and 4 cores here are my options: * AWS (t2.xlarge) $137.62 * OVH (EG-16) $79 * So You Start/OVH (SYS-IP-1S) $42 * Kimsufi/OVH (KS-3B) $29 * Hetzner (EX41) €46.41 [Lowest cost model is 32GB of ram] * Joe's Da…

Most of those providers you mentioned aren't as reliable and scalable as AWS, Google Cloud, Azure, etc. That isn't an apples to apples comparison.

I would not want to host my business on So You Start, Kimsufi, Hetzner, and especially not Joe's Data Center. I have personally used Joes DC and they have had numerous outages in the past. Hetzner is known for terminating you for any sort of "DOS" like traffic, including high PPS. To add to that, those providers (except maybe OVH?) don't let you scale up and down with automated APIs.

Re: A million-dollar engineering problem

#38

Post by sdflkd »

lossolo wrote: post #30
user5994461 wrote: post #22

Earlier quoted context omitted.

You shouldn't be on AWS in the first place if everything you do can fit on a single server. Use the right tool for the job.

I have services that do not fit on one server and still I don't need AWS. Distributing load/dividing services on x-xx servers is not rocket science, especially with tools we have available at the moment.

How does one begin to learn about these things? Minimizing cost of running services sounds super interesting but as a student I've never had to deal with it and am basically starting with 0 knowledge.

Re: A million-dollar engineering problem

#39

Post by Shivetya »

throwawaysed wrote: post #20
Negitivefrags wrote: post #16

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service i…

A lot of people are in denial that there isn't using some magic efficiency to cloud services that other datacenters don't have. Primary cost savings on cloud VM's is from overprovisioning. The more abstracted away the service is from the hardware, the more they can overprovision without customers noticing. The 50%+ profit margins have to be coming from somewhere. AWS is not made of magic, it's made from largely the s…

well to be honest its because it sounds cool, its in the news, and so how can such proposals be wrong. Seriously the number of times I heard "the cloud" bandied about by people who have continuously failed or were "thwarted" by people who knew better is getting silly. I am sure it happens in other places but too many think it makes them look smart to suggest it.

if anything its a diversion from fixing what is broke or even admitting to it within some organizations

Re: A million-dollar engineering problem

#40

Post by a13n »

teh wrote: post #6

I've been joking with friends that my next job will be AWS efficiency guru. I've somewhat optimized our own use, but I think I could use similar, simple rules to get 20% out of a 500k / month budget. Give me what I save you in 2 months and I'll have a good business :)

Or charge 20% of what you save them over the next year. This way you're charging more overall (especially if their costs are growing). Also your revenue will be more recurring rather than a one time thing. And by the time the 12 months is up, maybe they'll need your service again. :P
Post reply on HN