What are the performance numbers for work distributed like this?
Turning web traffic into a Super Computer
21–30 of 45 posts
Re: Turning web traffic into a Super Computer
#22Earlier quoted context omitted.
AWS at least already has very cheap ways to get compute time if you don’t particularly care about having uniform availability of it in the form of spot instances. The economics of people letting you use their browser doesn’t make sense. The pay is crap and the time when they are browsing is exactly when they would not want their computer doing a bunch of background processing. You’re better off letting people rent ou…
>AWS at least already has very cheap ways to get compute time No it does not. AWS and "very cheap" simply don't belong in the same sentence.
Compared to what? Useless to compare anything without numbers.
Re: Turning web traffic into a Super Computer
#23Earlier quoted context omitted.
AWS at least already has very cheap ways to get compute time if you don’t particularly care about having uniform availability of it in the form of spot instances. The economics of people letting you use their browser doesn’t make sense. The pay is crap and the time when they are browsing is exactly when they would not want their computer doing a bunch of background processing. You’re better off letting people rent ou…
Seti@home has been doing this for years
I was thinking of that exact use case — could this be a modern alternative that might get even wider distribution due to lower barriers to entry?
Re: Turning web traffic into a Super Computer
#24This is a great direction for a certain class of problems, but to call this a general approach for building a super computer ignores the biggest challenge of all: communication performance. The leadership-class super computers have large budgets for network interconnects like Infiniband, because the hardest problems require ultra low-latency, high-throughput RDMA message passing between processes.
Absolutely true. Embarrassingly parallel problems are generally not the most interesting, because you can just throw compute power at it until it is solved. Most applications running on actual supercomputers are communication bound.
Re: Turning web traffic into a Super Computer
#25I actually made this a few years ago for an ill-fated application to YC (I was still a sophomore in college).
The first big problem is that you can't trust any of the results that you receive from the workers. You can alleviate this somewhat by not accepting job results until they've been replicated by at least several other workers (lowering efficiency of the system), but then you're still vulnerable to sybil attacks.
The second issue was the total lack of privacy for the data and computations.
Re: Turning web traffic into a Super Computer
#26Earlier quoted context omitted.
>AWS at least already has very cheap ways to get compute time No it does not. AWS and "very cheap" simply don't belong in the same sentence.
Compared to what? Useless to compare anything without numbers.
Compared to what? The compute market in general.
AWS is a premium provider charging premium prices, they've essentially priced themselves as the most expensive provider on the market. It feels insane to describe them as "very cheap".
You might as well be calling a Bugatti Chiron "pretty cheap".
Any datacenter provider will be able to sell you compute time at a fraction of the cost of EC2. Look at OVH for example.
Re: Turning web traffic into a Super Computer
#27Ok, this is similar to Gridcoin (https://gridcoin.io/) , Golem (https://cryptoslate.com/coins/golem/) , SPARC (https://sparc.network/) , et al. I could keep going, but the point is, something like this exists. And we are seeing the issues of what happens when you embed a program that takes advantage of processing power. You get things like Coinhive (https://coinhive.com/), which were meant to be benign, turned into botnets.
Re: Turning web traffic into a Super Computer
#28Something like this could be a way to get finally rid of advertisements, and help finance free services in a novel new way if the CPU isn't over committed. I thought the same around crypto mining... Many people might prefer the resource providing/computation option compared to ads.
Good luck getting people to not double-dip.
Also good luck making it so I don't spend 10 cents of electricity or have my phone die early to give a site <1 cent of value.
Re: Turning web traffic into a Super Computer
#29I should code a fractal renderer that combines the work of many clients...
Re: Turning web traffic into a Super Computer
#30This could be a good alternative for SETI or that protein folding project but if it somehow turns into a plague of attempts to shift the computing requirements of running the backend onto the users, our content blocking lists will have to grow a little bit longer again.