Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What would you compute on 2000 badly behaved worker nodes?

news.ycombinator.com

Post reply on HN
Print view View original

21–29 of 29 posts

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#21

Post by joaojeronimo »

ris wrote: post #18
joaojeronimo wrote: post #14

Earlier quoted context omitted.

There is an opt-in: http://cdn.crowdprocess.com/opt-in.html (only one website requested it so it's not in English so far). It's as immoral as advertising, maybe even less. In advertising you show up at a web page and see tons of things that you did not want to see or did not bring you to that web page, sometimes shift your focus and annoys you. It's the same with CrowdProcess, except instead of annoying you, we annoy…

> It's as immoral as advertising, maybe even less. You've actually managed to convince yourself that, haven't you? It isn't because: 1: the user is paying for the electricity being wasted by you. A tab left open could be significantly detrimental. 2: it will cause real problems for mobile users who will be wondering why their battery's flat.

We -really- don't want to seem sketchy and immoral. We plan to stop computing after a certain amount of time (still to figure out, so far we don't have that many tasks running for it to be significant), and we completely block mobile phones and tablets while the Battery Status API isn't present in all devices (http://www.w3.org/TR/battery-status/ only Firefox implements it currently)

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#22

Post by patmcc »

joaojeronimo wrote: post #20
patmcc wrote: post #16

Earlier quoted context omitted.

>>We actually ask websites to tell they're a part of this, but we cannot control what they do because they can simply display:none. You could certainly just check to see they're using it properly. Do a screen scrape or even have someone hit the page every month or two, ban anyone abusing it by not notifying users.

that may be harder to do efficiently than building the entire platform and we're an extremely small team. I'm sure some day we'll do it but can't prioritize that now.

Really? How many websites are signed up / do you expect to sign up? Can you not spare 5 minutes per site per month to make sure there's a notification and/or opt-in? Or come up with an automated way to check it. Or use mechanical turk and pay somebody $0.50 per site to check for you.

If you can't prioritize something as important as running an ethical (and law abiding - take a very close look at the ramifications of unauthorized computer usage, which I think it could be argued you're doing with this platform) business, then you really shouldn't be in business.

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#23

Post by thaumaturgy »

joaojeronimo wrote: post #10
thaumaturgy wrote: post #8

Hmm. It's not clear from your documentation, is it possible to use xmlhttprequest through the web workers and get the response? Because having thousands of systems as distributed web crawlers would be really really cool.

It's not possible, sorry. It's possible if you ask us to access a certain address, but to the outside world it's not allowed. It would be pretty cool to have distributed web crawlers but it would also be extremely dangerous if someone decided to use CrowdProcess to do a DDoS.

Seems like that could be handled by your API, if you throttle requests by request domain. But, no worries, just curious.

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#24

Post by glimcat »

joaojeronimo wrote: post #15
pokoleo wrote: post #12

What you're saying is that you have a botnet. Look at botnet owners.

That's actually pretty good advice, except it may be hard to find legal and moral things to run. We wanted to find the cure for cancer and not produce rainbow tables or do DDoS attacks.

It's also hard to find legal and moral justification for constructing a botnet in the first place.

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#25

Post by lucastx »

patio11 wrote: post #4
pedrombafonso wrote: post #3

Earlier quoted context omitted.

Thanks for your answer! Do you know about any real life application for that problem?

If you're freezing to death inside of a computer cluster n-queens will save your life.

laughing out loud, here.

took me some seconds to understand this subtle one.

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#26

Post by pgfonseca »

patmcc wrote: post #22
joaojeronimo wrote: post #20

Earlier quoted context omitted.

that may be harder to do efficiently than building the entire platform and we're an extremely small team. I'm sure some day we'll do it but can't prioritize that now.

Really? How many websites are signed up / do you expect to sign up? Can you not spare 5 minutes per site per month to make sure there's a notification and/or opt-in? Or come up with an automated way to check it. Or use mechanical turk and pay somebody $0.50 per site to check for you. If you can't prioritize something as important as running an ethical (and law abiding - take a very close look at the ramifications of…

It isn't as much a matter of ability to verify, but to enforce. Currently the platform is supplied by quite a few websites (over 100), and the best way to get them to adequately communicate this is through proper incentives.

The incentives must be: if you do not comply, your content won't be monetized (as would happen with ads).

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#28

Post by joaojeronimo »

glimcat wrote: post #24
joaojeronimo wrote: post #15

Earlier quoted context omitted.

That's actually pretty good advice, except it may be hard to find legal and moral things to run. We wanted to find the cure for cancer and not produce rainbow tables or do DDoS attacks.

It's also hard to find legal and moral justification for constructing a botnet in the first place.

Well we noticed that flash ads sometimes take up even more than 100% of a CPU (meaning it can spawn threads and use multicore processing), video ads perhaps even more since they may get to be gpu accelerated, as CSS3 animations. We figured if people are spending this much CPU cycles for advertising, than why not clean up all the advertising and use the CPU cycles for some protein folding and finding a cure for cancer to make a website owners, visitors and a group of researchers happy ?

Re: Ask HN: What would you compute on 2000 badly behaved worker nodes?

#29

Post by joaojeronimo »

apw wrote: post #27

I checked the FAQ, but didn't see an answer -- how do you prevent malicious actors from returning bogus data?

We thought of sending puzzles to the worker that would have to really be computed by the VM and would take some amount of time until it was possible to be faked, or would have changed by the time a human could decipher the puzzle and return the expected result, but so far we're only ignoring bad actors and sometimes comparing results from different actors until a quorum is found among results
Post reply on HN