Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A million-dollar engineering problem

segment.com

Post reply on HN
Print view View original

251–260 of 265 posts

Re: A million-dollar engineering problem

#251

Post by hueving »

edanm wrote: post #208
hueving wrote: post #202

Earlier quoted context omitted.

Gross. AWS is becoming the Cisco of cloud computing. Get people hooked when they are low revenue and twist the screws in later when they get big.

In what way is this gross? They're giving people what is essentially a loan, which they are uniquely able to give, to help them start a business, in the hopes that this will be an even bigger business. This is almost the definition of a win-win. Customers are free to leave at any time or work with other providers, but they choose AWS because they give them more value.

It's not a loan. They don't require any repayment. It's a discount.

>This is almost the definition of a win-win.

Not long-term for the company that gets lured into this trap. Just like with Cisco/Oracle, companies will waste millions of dollars once they are cornered into a situation like this because the switching costs are so high even though there are plenty of other alternatives.

Re: A million-dollar engineering problem

#252

Post by thecrazyone »

hinkley wrote: post #203
kafkaesq wrote: post #116

Earlier quoted context omitted.

A blanket "never use AWS" isn't going to address it constructively. But that's not what the commenter was saying, of course. More like (paraphrasing) "before you go full-hog on AWS, do some simple math first." Which an amazingly high percentage of people neglect to do, these days.

Weeks of work can save hours of meetings.

interesting thing to say, but did you plan for it to mean something in this context? :)

Re: A million-dollar engineering problem

#253

Post by kafkaesq »

thecrazyone wrote: post #252
hinkley wrote: post #203

Earlier quoted context omitted.

Weeks of work can save hours of meetings.

interesting thing to say, but did you plan for it to mean something in this context? :)

Nothing special - just the whole "Yeah, saw your email† ... look, I'm so busy, did you get the site up and running again? I'll get back to you on this after my vacation" management style which seems to drive a great many "decisions" that are made in this realm. And on which AWS, implicitly, banks a great deal of its business model on.

† About the monthly you-know-what bill now well into 5 figures, and climbing.

Re: A million-dollar engineering problem

#254

Post by sillysaurus3 »

sdflkd wrote: post #248
sillysaurus3 wrote: post #134

Earlier quoted context omitted.

I worked through the labs at https://pdos.csail.mit.edu/6.824/ for fun. It's more along the lines of "How can we write a distributed fault-tolerant database?" but you might like it anyway. Lab 4 is a beast with more lives than . The moment you think you've finally written your distributed system correctly, the unit tests will prove your service fails during XYZ network partition topologies. It's very worthwhile to be…

Wrt the first link, I'm going to be taking a distributed systems course next semester that (hopefully) covers the same material. Nice to know what I learn at school is somewhat applicable. My question is learning about design ideas like having Redis keeping a list of "work to be done" and such. Using modern tools to combat 'modern' problems. Is it just something you figure out after knowing the fundamentals (i.e. whe…

Yeah, pretty much. I've never actually done it, but I know I could do it if I needed to.

If it sounds mysterious, think of it this way: Imagine you were thrown into a room with a computer, internet, and endless food, and the only way out of the room was to solve this problem. I bet you'd figure it out within a week or two, or a couple months max. (If only to go have a shower.)

One thing to watch out for: Solving modern problems can be pretty unsatisfying. Before you experience what life is like at a modern company, you tend to think of them as functional, orderly, and planned. Real companies are almost universally dysfunctional, disorderly, and haphazard. It's very rare that a plan is conceived, followed, and deployed without growing some warts and throwing in some hacks to get it done.

So I think you should enjoy this time, where you're free to think of thought-problems like "What's the most correct and extensible way to solve this problem?" instead of being forced to solve them on a time crunch.

Re: A million-dollar engineering problem

#255

Post by kierenj »

wazoox wrote: post #240

A friend of mine developed something to do just that: understand AWS/Google/Azure costs and compare prices (and save big). Shameless plug: https://trackit.io/

Probably not the ideal response, sorry, but I only got as far as the homepage - really slow, jerky scrolling, scroll-to-anchors miss the mark, and the social media links don't work. Then, mouse wheel/trackpad scrolling stopped working. :/

Re: A million-dollar engineering problem

#256

Post by geezerjay »

tiplus wrote: post #217
gkop wrote: post #141

Earlier quoted context omitted.

So how many does it buy?

I would estimate the cost for three years like this: - 1U single socket Supermicro Xeon E5v4-2630 10core+1GTX1070 GPU = 1500EUR - operation costs (power+cooling) over 3 years = 1500EUR - cost of 2 hardware admins 160k/year - renting local rack space 15EUR/U This equals about 145 servers for 3 years for us for 1 Million. The setup is highly optimized for _one_ set of data crunching tasks, not DB. We do not need much r…

>cost of 2 hardware admins 160k/year

This is by far the most expensive item in your estimate, and I'm sure that you don't suddenly need to hire another pair of admins when you buy a single Xeon blade.

Once you factor this fact in your estimates, you'll realise that the unit cost of each server is, unlike your estimate, negligible.

Re: A million-dollar engineering problem

#257

Post by geezerjay »

patrickg_zill wrote: post #231
walrus01 wrote: post #87

Earlier quoted context omitted.

A million dollars buys a shitload of fast 1U dual socket xeon servers each with 512GB of RAM.

There are companies that will lease you a dedicated server with 40 cores and 256gb RAM for $499/month. Figure a 10% quantity discount and you are looking at at least 200 of those for about $1 million per year.

Just because a company is ripping you off, it doesn't mean that's representative of the real market.

For 500€ a month, it's possible to buy your own 40 core blade with 256gb of RAM. In fact, it's possible to buy a pair of those servers.

Re: A million-dollar engineering problem

#258

Post by QuinnyPig »

rdtsc wrote: post #191
QuinnyPig wrote: post #51

Earlier quoted context omitted.

> At this point simply not using AWS is a competitive advantage. Respectfully, I'm going to disagree. I consult full time on AWS cost optimization / reduction / understanding. If you blindly run things on AWS without an understanding of the costing model, that'll work for a time. As you scale, you start to realize "oh my god it runs on money." There are myriad ways around that, but a blanket "never use AWS" isn't goi…

> I consult full time on AWS cost optimization / reduction / understanding. Curious, have you ever consulted a client not to use AWS? As in "your cost optimization is going to be not using AWS and using $x instead"?

Absolutely. Great example: container workloads without high data volumes. Slap those into GCP if you can-- or arbitrage between different cloud providers / regions / on-prem if you have them.

That said, clients who come to me already have a significant investment with AWS. Getting rid of that is a high bar in terms of engineering effort.

That said, I've no financial incentive to recommend one provider over another. My clients pay me, the vendors do not.

Re: A million-dollar engineering problem

#259

Post by QuinnyPig »

mrlatinos wrote: post #234
rdtsc wrote: post #191

Earlier quoted context omitted.

> I consult full time on AWS cost optimization / reduction / understanding. Curious, have you ever consulted a client not to use AWS? As in "your cost optimization is going to be not using AWS and using $x instead"?

Curious about this as well. Otherwise, it's like a car salesman saying you shouldn't buy a bike, rather, understand mpg before buying a car.

Sure-- but I'm usually not retained until the environment is built out. The client's starting point is relevant to what comes next.

Re: A million-dollar engineering problem

#260

Post by wazoox »

kierenj wrote: post #255
wazoox wrote: post #240

A friend of mine developed something to do just that: understand AWS/Google/Azure costs and compare prices (and save big). Shameless plug: https://trackit.io/

Probably not the ideal response, sorry, but I only got as far as the homepage - really slow, jerky scrolling, scroll-to-anchors miss the mark, and the social media links don't work. Then, mouse wheel/trackpad scrolling stopped working. :/

THe website is probably hosted on AWS :) As for the social media links, they all work for me...
Post reply on HN