Thanks HN: Lessons learned after Google nearly killed my site • Hacker News Board

Re: Thanks HN: Lessons learned after Google nearly killed my site

#241

Post by AnthonyMouse » Sun, Mar 07, 2021, 4:10 AM UTC

unishark wrote: post #196

rvnx wrote: post #190

Earlier quoted context omitted.

The job of the police

Police take reports from victims then what? How do police protect you from a site hosted outside their jurisdiction? I think tech companies deciding what people can access is the most likely endgame no matter what. People will demand protection. Whether it's a great firewall, a whitelist-only internet, ...or just automated filtering like this, which may be the most liberal option we can realistically expect.

The police go and arrest them. This is true even for other jurisdictions. It's not as if there are no police in France or Australia.

What you're left with is things hosted out of uncooperative countries like Russia or China. But then shouldn't the block list consist entirely of things hosted out of uncooperative countries like Russia or China? How did this US-hosted business fall victim?

Re: Thanks HN: Lessons learned after Google nearly killed my site

#242

Post by brongondwana » Sun, Mar 07, 2021, 4:17 AM UTC

core-utility wrote: post #176

that_guy_iain wrote: post #86

>Now we run automated tests to monitor server uptime and check server for problems every 30 seconds. Unfortunately automated test scripts were happily getting HTTP/200 replies while people using the Chrome browser were being told this is a scam business trying to steal their bank account information. I was surprised this wasn't part of the lessons learned. But it seems the monitoring basically failed but that wasn't…

When talking about checks on the order of twice a minute, curl is probably the right approach. You can/should still do a full check, but that can be done at a lower frequency.

Yep, for sure. Fastmail still uses a once-every-10-minutes frequency for the full end-to-end "can log into the website, compose an email, receive the email, trigger an automated background fetch, receive that email too" tests, though the "can connect to service" tests run much more often.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#243

Post by klingon79 » Sun, Mar 07, 2021, 4:19 AM UTC

sterlind wrote: post #211

scrollaway wrote: post #175

Earlier quoted context omitted.

Frankly, Google's Safe Browsing list is one of those things which should be broken out of Google and ran by an independent entity, much like the Let's Encrypt model. I don't know how to help make this happen.

I'd suggest: 1. Mozilla and the Chrome clones (Edge, Brave, etc.) partner to make an (open as possible) standard for blocking and reviewing, and start maintaining their own list upstream from Google. 2. When Google adds to their blocklist, independently check it according to the consortium's own standards. 3. Cut a deal with Bing or Yandex to scan for malware as part of their crawls, to get technology independence. 4…

I still don’t understand what the problem is here.” With Google doing this.

We don’t know that this whole thing is not invented by the author or that they were indeed not doing anything malicious.

All of this, as credible as it may seem, could just be invented bad PR against Google, which if true should make us take a hard look at whomever is behind this.

Seriously, read the Lessons Learned again and tell me for certain this really happened. How in the world would it look so staged, with the bold words and thought-out structure. I’m not going to jump on the bandwagon everytime someone makes claims.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#244

Post by AnthonyMouse » Sun, Mar 07, 2021, 4:23 AM UTC

ttt0 wrote: post #238

AnthonyMouse wrote: post #235

Earlier quoted context omitted.

But regulation is being proposed as an alternative to competition. It can't do that. You need competition whether or not you have regulation. Look at what a dumpster fire every regulated private monopoly is. "Success" is a stagnant inefficient bureaucracy that ossifies everything it touches. Failure is the F-35 wasting a number of public dollars with twelve zeroes after it.

As a solution to this particular problem, not banking overall. And we're not talking about central banks, but normal banks. The competition is still there. Tell me, why shops are not allowed to discriminate on the basis of race? Why not just have competition deal with that?

> As a solution to this particular problem, not banking overall.

Competition would work as a solution to this particular problem and overall.

> And we're not talking about central banks, but normal banks. The competition is still there.

Banks have notoriously high switching costs and barriers to entry. What competition would look like is a regulatory environment that enabled switching banks to be as easy as switching wireless providers after number portability.

> Tell me, why shops are not allowed to discriminate on the basis of race? Why not just have competition deal with that?

Originally, because white customers would refuse to patronize shops that served black customers, effectively acting as a monopsony (lack of competition) and the law was needed to restore competition.

Today, mostly for historical reasons. Do you honestly think that a black person right now would have trouble finding a restaurant to eat in even if there was no law forcing all restaurants to serve them?

Re: Thanks HN: Lessons learned after Google nearly killed my site

#245

Post by exikyut » Sun, Mar 07, 2021, 4:40 AM UTC

noisy_boy wrote: post #227

that_guy_iain wrote: post #86

>Now we run automated tests to monitor server uptime and check server for problems every 30 seconds. Unfortunately automated test scripts were happily getting HTTP/200 replies while people using the Chrome browser were being told this is a scam business trying to steal their bank account information. I was surprised this wasn't part of the lessons learned. But it seems the monitoring basically failed but that wasn't…

Why not run a fully functional monitoring job i.e. upload file with a monitoring account and check the results to validate that it is working end to end. Doing this even once a minute shouldn't put any load and is a much more reliable test.

It would have needed to be done through Chromium in non-headless (likely full Xvfb) mode, with step screenshots, and screenshot comparison (always flaky!), for the Safe Browsing interstitial to have generated an alert.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#246

Post by ttt0 » Sun, Mar 07, 2021, 4:42 AM UTC

AnthonyMouse wrote: post #244

ttt0 wrote: post #238

Earlier quoted context omitted.

As a solution to this particular problem, not banking overall. And we're not talking about central banks, but normal banks. The competition is still there. Tell me, why shops are not allowed to discriminate on the basis of race? Why not just have competition deal with that?

> As a solution to this particular problem, not banking overall. Competition would work as a solution to this particular problem and overall. > And we're not talking about central banks, but normal banks. The competition is still there. Banks have notoriously high switching costs and barriers to entry. What competition would look like is a regulatory environment that enabled switching banks to be as easy as switching…

> Do you honestly think that a black person right now would have trouble finding a restaurant to eat in even if there was no law forcing all restaurants to serve them?

No. My point is that it's not as big of a deal as you make it out to be. Competition didn't suddenly disappear because of that.

In the EU we already have the laws we're talking about, requiring banks to provide you a basic bank account. The competition between banks didn't disappear here either.

Also banking is not exactly a free market one way or another.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#247

Post by frutmonkey » Sun, Mar 07, 2021, 5:00 AM UTC

dhosek wrote: post #240

lostcolony wrote: post #221

Earlier quoted context omitted.

I think if Google is going to decide to police the web like this, they need to alert people more proactively. The first the OP should have heard of this was Google emailing them through uploader.win's Contact Us email address. It's easy and obvious to find on the site; seems like that should be part of the automated process.

Locating contact info for websites is not something that can be automated. Some sites provide an email address, some a form, some point people at twitter or facebook and some don't provide any contact information at all. None of this is arranged in any sort of standard way. Contact info may be under a link marked "contact" or "about" or "bio" or appear at the bottom of every page.

Legally, every web domain must have a contact point when you register. They shouldn't have to webcrawl for the contact info. Then again, they are already the kings of webcrawling....

Re: Thanks HN: Lessons learned after Google nearly killed my site

#248

Post by lostcolony » Sun, Mar 07, 2021, 5:00 AM UTC

frutmonkey wrote: post #247

dhosek wrote: post #240

Earlier quoted context omitted.

Locating contact info for websites is not something that can be automated. Some sites provide an email address, some a form, some point people at twitter or facebook and some don't provide any contact information at all. None of this is arranged in any sort of standard way. Contact info may be under a link marked "contact" or "about" or "bio" or appear at the bottom of every page.

Legally, every web domain must have a contact point when you register. They shouldn't have to webcrawl for the contact info. Then again, they are already the kings of webcrawling....

Yeah; that was my original thought, but not sure how readily available that is without reaching out to the registrar. Certainly, the registrar could be reached out to.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#249

Post by lostcolony » Sun, Mar 07, 2021, 5:02 AM UTC

dhosek wrote: post #240

lostcolony wrote: post #221

Earlier quoted context omitted.

I think if Google is going to decide to police the web like this, they need to alert people more proactively. The first the OP should have heard of this was Google emailing them through uploader.win's Contact Us email address. It's easy and obvious to find on the site; seems like that should be part of the automated process.

Locating contact info for websites is not something that can be automated. Some sites provide an email address, some a form, some point people at twitter or facebook and some don't provide any contact information at all. None of this is arranged in any sort of standard way. Contact info may be under a link marked "contact" or "about" or "bio" or appear at the bottom of every page.

Certainly. If only Google was good at making sense of web page content.

Point isn't they need to do it for every site. Just, a semblance of effort seems like the kind of thing that's worth throwing some resources at.

Re: Thanks HN: Lessons learned after Google nearly killed my site

#250

Post by shadowgovt » Sun, Mar 07, 2021, 5:07 AM UTC

dhosek wrote: post #240

lostcolony wrote: post #221

Earlier quoted context omitted.

I think if Google is going to decide to police the web like this, they need to alert people more proactively. The first the OP should have heard of this was Google emailing them through uploader.win's Contact Us email address. It's easy and obvious to find on the site; seems like that should be part of the automated process.

Locating contact info for websites is not something that can be automated. Some sites provide an email address, some a form, some point people at twitter or facebook and some don't provide any contact information at all. None of this is arranged in any sort of standard way. Contact info may be under a link marked "contact" or "about" or "bio" or appear at the bottom of every page.

Hypothetically, every domain is supposed to have contact information attracted to it.

Unfortunately, that was abused.