Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scalability, but at what cost? (2015)

frankmcsherry.org

Post reply on HN
Print view View original

21–30 of 80 posts

Re: Scalability, but at what cost? (2015)

#21

Post by collyw »

brianwawok wrote: post #9
wueiued wrote: post #7

I think graph operations are not fair comparison. It is notoriously difficult to scale. On other side AWS now offers 2TB RAM machine. And single huge machine has smaller per GB cost than several smaller machines. I think clustered computing as we know will be soon gone. Only reason for multiple machines will be availability.

Do you think our datasets will stop growing? It seems to me that data is growing faster than RAM, and has been for years. How do we find the upper limit of data? The Human Genome is finite, it will only get so big. What you did on facebook? Seems near infinite....

I can't remember where it was I saw it recently, but it was pointed out that there are diminishing returns on using more data. Your accuracy with something like Google's pagerank is good with 1 billion data points for input, but doesn't improve much with more data.

Re: Scalability, but at what cost? (2015)

#22

Post by vonmoltke »

ChuckMcM wrote: post #10

I like the analysis, basically it says "hey you don't have big data" :-) but that requires a bit more explanation. The only advantage of clustered systems like Spark, Hadoop, and others is aggregate bandwidth to disk and memory. We know that because Amdahl's law tells us that parallelizing something invariably adds overhead. So from a systems perspective that overhead has to be "paid for" by some other improvement, a…

> The only advantage of clustered systems like Spark, Hadoop, and others is aggregate bandwidth to disk and memory.

No it isn't. There are plenty of CPU-bound tasks that run much faster if the work is distributed in parallel across multiple machines. We use Hadoop at my company primarily for that reason.

Re: Scalability, but at what cost? (2015)

#23

Post by nostrebored »

Joeri wrote: post #17
ChuckMcM wrote: post #10

I like the analysis, basically it says "hey you don't have big data" :-) but that requires a bit more explanation. The only advantage of clustered systems like Spark, Hadoop, and others is aggregate bandwidth to disk and memory. We know that because Amdahl's law tells us that parallelizing something invariably adds overhead. So from a systems perspective that overhead has to be "paid for" by some other improvement, a…

For a 2 TB dataset you could also pay supermicro 50k to get a 40 core 3 TB RAM monster that can keep that whole dataset in RAM. At 50 GB / sec throughput that would keep your query roundtrip time at somewhere around the minute mark. Not quite 3 seconds, but then not quite a thousand nodes either. Of course, rebooting that machine would be awkward. Still, I think the general rule applies that if you can buy a server t…

How about when that operation is at the core of your business logic and happens multiple time a day? How about when it's a blocking operation?

There are plenty of cases where your rule doesn't hold true, and I don't think that they're that uncommon.

Plus having many machines has the advantage of a higher granularity of control over workload and balancing.

Re: Scalability, but at what cost? (2015)

#25

Post by sigusr1 »

brianwawok wrote: post #3
3pt14159 wrote: post #2

I've been doing data analysis and machine learning client work for quite some time now and for companies as small as a 3 person startup to advising a department of the Canadian government. Almost always numpy matrix math + cython or C or Java on a single machine is enough. Not always-always; but if you can relax requirements slighly say by accepting a 45 minute lag from new data impacting the total model, or by cachi…

This is kind of the same argument as microservices. We could write 30 microservices deployed on 30 docker images with load balancing and FT and all that magic for a basic webapp... Or we could just write a pretty fast webserver and do it with 1 server. (Or if it is stateless, do it with a few for still a lot less work than a giant microservice cluster). I think in the last year or so microservices have become a littl…

If you haven't seen stackoverflows server architecture posts you'd probably enjoy them: https://nickcraver.com/blog/2016/02/17/stack-overflow-the-ar...

I still think containers are great, and I really like the abstractions kubernetes provides, but if I ever had enough traffic to worry about scaling, I envision running a small cluster of very powerful computers rather than 100s of weak ones.

Re: Scalability, but at what cost? (2015)

#26

Post by JoachimSchipper »

Joeri wrote: post #17
ChuckMcM wrote: post #10

I like the analysis, basically it says "hey you don't have big data" :-) but that requires a bit more explanation. The only advantage of clustered systems like Spark, Hadoop, and others is aggregate bandwidth to disk and memory. We know that because Amdahl's law tells us that parallelizing something invariably adds overhead. So from a systems perspective that overhead has to be "paid for" by some other improvement, a…

For a 2 TB dataset you could also pay supermicro 50k to get a 40 core 3 TB RAM monster that can keep that whole dataset in RAM. At 50 GB / sec throughput that would keep your query roundtrip time at somewhere around the minute mark. Not quite 3 seconds, but then not quite a thousand nodes either. Of course, rebooting that machine would be awkward. Still, I think the general rule applies that if you can buy a server t…

Idle question: is 50 GB/s a reasonable throughput to expect for such a monster? DDR4 has a peak transfer rate of ~12.8-19.2 GB/s per stick (per https://en.wikipedia.org/wiki/DDR4_SDRAM), so I'd expect quite a bit more bandwidth for predictable accesses - are you using some useful rule-of-thumb, or just unduly pessimistic? ;-)

Re: Scalability, but at what cost? (2015)

#27

Post by ChuckMcM »

Joeri wrote: post #17
ChuckMcM wrote: post #10

I like the analysis, basically it says "hey you don't have big data" :-) but that requires a bit more explanation. The only advantage of clustered systems like Spark, Hadoop, and others is aggregate bandwidth to disk and memory. We know that because Amdahl's law tells us that parallelizing something invariably adds overhead. So from a systems perspective that overhead has to be "paid for" by some other improvement, a…

For a 2 TB dataset you could also pay supermicro 50k to get a 40 core 3 TB RAM monster that can keep that whole dataset in RAM. At 50 GB / sec throughput that would keep your query roundtrip time at somewhere around the minute mark. Not quite 3 seconds, but then not quite a thousand nodes either. Of course, rebooting that machine would be awkward. Still, I think the general rule applies that if you can buy a server t…

I think this is also a vary salient observation. It is always possible to construct really large systems, right up to mainframes. And if you're asked to do so it helps to take as wide a view on the problem as you can.

So from a technical point of view, 50GB / sec on a single machine vs 600GB / sec on a 1000 node cluster. From a cost perspective, running 1 machine is going to be a lot less than running 1,000 machines.

Consider some other aspects as well. If a machine breaks, and you have only one, you are offline, if one breaks and you have 999 left, you're still 99.9% up and running. If you work in 2TB data sets, how many do you have? One? Two? Twenty? The more you have the more storage you end up putting on a machine, and even with a SAN the ability to move terabytes around is a pain. Then there is the enterprise value of the analysis. How much does the analysis add value to the product you sell? In the paper's example of Page Rank one could argue it really made Google's engine better so a lot of value. In an oil and gas context it might be the difference between finding oil or not, so again high value. But in a twitter 'bot' analysis, killing off all the identified bot accounts might have very little relative value to the overall business.

The bottom line is that none of these sorts of choices can be made in isolation. Looking at the choices through a single lens, whether it is performance, cost, or capability, is rarely sufficient to make the best choice. What is more the best choice may seem like a "bad" choice from an engineering perspective but great from a finance perspective. Similarly a good choice from a finance perspective could be a horrible choice from an engineering perspective.

What is important is to keep in mind the strengths of the various choices available to you, and their weaknesses. Then to select from them based on the current and future requirements for the resulting system.

Re: Scalability, but at what cost? (2015)

#28

Post by herge »

dzdt wrote: post #14

This reminds me of the "your data fits in ram" website which was on HN last year. Basically that site asked you your data size, then answered "yes" for any size up to a few TB. The website is down, but the HN discussion is still there : https://news.ycombinator.com/item?id=9581862 . In fact the top comment there links to the original post here.

> The website is down,

Maybe they ran out of ram?

Re: Scalability, but at what cost? (2015)

#29

Post by Joeri »

nostrebored wrote: post #23
Joeri wrote: post #17

Earlier quoted context omitted.

For a 2 TB dataset you could also pay supermicro 50k to get a 40 core 3 TB RAM monster that can keep that whole dataset in RAM. At 50 GB / sec throughput that would keep your query roundtrip time at somewhere around the minute mark. Not quite 3 seconds, but then not quite a thousand nodes either. Of course, rebooting that machine would be awkward. Still, I think the general rule applies that if you can buy a server t…

How about when that operation is at the core of your business logic and happens multiple time a day? How about when it's a blocking operation? There are plenty of cases where your rule doesn't hold true, and I don't think that they're that uncommon. Plus having many machines has the advantage of a higher granularity of control over workload and balancing.

Hey, I'm building a hadoop system myself, so I know there are workloads where the rule doesn't apply. Like all rules there are exceptions.

Re: Scalability, but at what cost? (2015)

#30

Post by brianwawok »

aianus wrote: post #16
ben_jones wrote: post #8

Earlier quoted context omitted.

People forget that the two methods of scaling are HORIZONTAL and VERTICAL. They think: "I can just put some micro-services behind HA proxy and boom, more capacity!". And then they forget if they had just modified that one query and tweaked that one for-loop they could've had that same capacity without launching six new servers with all kinds of potential for the wiring to go down and cause downtime. Plus the dev time…

Vertical scaling requires hiring good engineers instead of mediocre ones (additional cost of $100,000s per year across the team). Horizontal scaling in comparison is much, much cheaper for your average CRUD app.

Maybe, maybe not.

For example choosing Java over Ruby would give you 2-10x better perf per server... and I am not sure that Java devs cost any more or less than Ruby devs.

Now we can get into an argument about developer productivity and all that.. but form a purely "i want to run 10x more users per server"... something like Java / Ruby gets you a long ways.

Post reply on HN