Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A million-dollar engineering problem

segment.com

Post reply on HN
Print view View original

101–110 of 265 posts

Re: A million-dollar engineering problem

#101

Post by user5994461 »

kornish wrote: post #98
user5994461 wrote: post #84

Earlier quoted context omitted.

> the per-user ID write bottleneck The basics is to have less machines but more powerful. It helps to handle the targeted bursts. The advanced is to have a layer of queuing before the ingestion, where you can do magic with distribution rules, rate limiting and dropping peak traffic.

Yep, you're absolutely right. In multitenant SaaS apps with extremely uneven distribution of traffic, it's pretty common for large customers to get their own dedicated DB servers. > The advanced is to have a layer of queuing before the ingestion, where you can do magic with distribution rules, rate limiting and dropping peak traffic. And batch loading – don't forget batching!

I didn't mean to have special databases, that is another level of optimization.

I mean to have bigger servers for everything. For instance, a farm of 4x 10cores servers is likely to process data more consistently than 10x 4cores servers.

Re: A million-dollar engineering problem

#102

Post by dcancel »

kornish wrote: post #70
burntwater wrote: post #68

Had an interesting experience when I went to the pricing page for Segment - Had a little sales guy pop up in the lower right corner asking "Would (my enterprise employer name) like to know if you qualify for a volume discount?" I only recently started working (and web browsing) from an enterprise network, so maybe this has been done before, but it's my first time seeing it and I thought it was really interesting. Edi…

Did the box have a "Powered by x" statement anywhere? Would be curious to know if it's Intercom or something else.

It's Drift (www.drift.com). I'm the CEO. We work closely with Segment and others including Clearbit to personalize the experience.

Re: A million-dollar engineering problem

#104

Post by dcancel »

davidlumley wrote: post #83
ropiku wrote: post #81

Earlier quoted context omitted.

I don't get it right now but it's possible that is Intercom (chat) with data from Clearbit ( https://clearbit.com/reveal ).

Close! It's Drift (drift.com), not Intercom.

Thanks David.

You're right it's Drift. I'm the CEO. Love working with Segment, Mixpanel, Clearbit and other great companies to deliver a personalized experience on their sites like in this example.

Re: A million-dollar engineering problem

#105

Post by alexkon »

QuinnyPig wrote: post #51
Negitivefrags wrote: post #16

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service i…

> At this point simply not using AWS is a competitive advantage. Respectfully, I'm going to disagree. I consult full time on AWS cost optimization / reduction / understanding. If you blindly run things on AWS without an understanding of the costing model, that'll work for a time. As you scale, you start to realize "oh my god it runs on money." There are myriad ways around that, but a blanket "never use AWS" isn't goi…

How do you approach a horrifying AWS bill? Can you recommend any reading? I’m going to need to optimize a tiny fleet on GCE pretty soon.

Re: A million-dollar engineering problem

#106

Post by bearjaws »

I work in a start up, we own all our own hardware, and it is HELL.

We are forced to pay extremely large sums of money to upgrade our infrastructure as any purchase requires a redundant piece as well.

For example we have used 90% of our SANs storage, our IO is suffering and now were looking at purchasing two $10k SANs to upgrade. In the meantime, we have probably spent over 10k worth of development time to compress, clean up, and remove any data we don't need. We will not get the weeks of development effort back, we will continue to spend more money on hardware, and we will continue to suffer to delay projects because of poor choices.

If we had AWS, we could have written an entire sprints worth of work (also ~$14k worth of IT time) and moved on with our lives.

Clearly, some companies let the budget keep growing, but for most companies AWS flexibility is worth its weight in gold.

Re: A million-dollar engineering problem

#108

Post by divbit »

divbit wrote: post #63
divbit wrote: post #54

Idk how common the setup in the article is, but seems they could easily test for heatmap singularities such as the one that happened and alert for that.

(just alert any keys with heat > mean +- x *standard deviation or something stupid like that)

Actually if you have 3d features, could use edge detection algorithm like Kirsch after fuzzing the colors a bit.

Re: A million-dollar engineering problem

#109

Post by user5994461 »

walrus01 wrote: post #87
Negitivefrags wrote: post #16

A friend of mine was annoyed that a small service he liked was shutting down. He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making. They were spending 5k a month on AWS crap and claimed it was impossible to get any lower. He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service i…

A million dollars buys a shitload of fast 1U dual socket xeon servers each with 512GB of RAM.

-looks at this datacenter-

Well, actually it doesn't.

RAM and Xeons are expensive.

Re: A million-dollar engineering problem

#110

Post by user5994461 »

QuinnyPig wrote: post #52
teh wrote: post #6

I've been joking with friends that my next job will be AWS efficiency guru. I've somewhat optimized our own use, but I think I could use similar, simple rules to get 20% out of a 500k / month budget. Give me what I save you in 2 months and I'll have a good business :)

I actually do this as a full time thing; I started a consultancy to fix horrifying AWS bills. Something I've learned is that flat fee pricing makes the most sense-- while tempting, the other models are off-putting. Hourly is a great way to starve to death, and "percentage of savings" grows difficult to quantify. "Okay, you just recommended the following reserved instance purchases. Is this really the best for us, or…

Put a maximum amount to the fee. Let's say 100k for the month I am here.
Post reply on HN