Live data from Hacker News

My history of (mostly failed) side projects and startups

gabrielweinberg.com

51–60 of 82 posts

Re: My history of (mostly failed) side projects and startups

#51

Earlier quoted context omitted.

Here's my (completely unsubstantiated) theory. It happend literally the day after crossing $500 & 50K views in adsense. I'm guessing one of those was a trigger for manual review by some contractor, perhaps overseas. They looked at my sites for 3 sec, found them to be cookie-cutter and decided to blacklist the account. I get the impression they shoot first, ask questions later. I didn't feel like dealing with it all o…

I talked to someone from Google at I/O who should know and he claimed they don't play "Whack a mole" with websites. They will tweak their ranking algorithm to punish the behavior they see in a web site they don't want to be ranked.

That was probably me. We have two sides to the webspam team at Google: engineering and manual. We definitely prefer to write algorithms so that we avoid dealing with individual websites--the idea is that you strive to fix the root cause of an issue, not to tackle specific sites. However, if we see a website that violates our guidelines and that gets past the algorithms, we are willing to take manual action. Where possible, we use the output of the manual team not only to reduce spam itself, but to train the next iteration of algorithms.

For example, one of the big issues in blackhat spam this past year was illegally hacked sites. Our algorithms weren't doing the best job on hacked sites, so the manual team kept an eye out for hacked sites to remove them (and often to alert the website owners that they'd been hacked). The data generated by the manual team helped us build and deploy multiple new algorithms to detect hacked sites, leading to a 90% reduction in the number of hacked sites showing up in Google's search results in the past few months. That decrease in hacked spam in turn frees up the manual team to tackle the next bleeding-edge technique the spammers use.

I suspect every major search engine uses similar approaches: try to stop the majority of spam with algorithms, but be willing to take action in the mean time while engineers work to improve the algorithms.

Re: My history of (mostly failed) side projects and startups

#52

Why did Google blacklist all of your Tldscan sites? Was it just because your sites' content was updated automatically? Or was it because you did something wrong for SEO?

Here's my (completely unsubstantiated) theory. It happend literally the day after crossing $500 & 50K views in adsense. I'm guessing one of those was a trigger for manual review by some contractor, perhaps overseas. They looked at my sites for 3 sec, found them to be cookie-cutter and decided to blacklist the account. I get the impression they shoot first, ask questions later. I didn't feel like dealing with it all o…

I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links.

I mean, if you're auto-generating a page that has this text: "Elcorillord.com

Common misspellings and typos: Elcroillord.com, Elcorilolrd.com, Elcori.lord.com, Elcofillord.com, www.elcorillord.com, lEcorillord.com, Elcotillord.com, Elcorillor.com, Elcoeillord.com, Elcori,lord.com, Wlcorillord.com, Elcorjllord.com, Elcorillord.coom, Elocrillord.com, Elcor8llord.com, Elvorillord.com, Elcorillprd.com, Elcorillord.cim, Elcorillorf.com, Elcorilloed.com, Elorillord.com, Elckrillord.com, Elcoriplord.com, Elcorillord.ckm, Elcorillord.cm, Elcorillord.ccom, Epcorillord.com, Elcoril;ord.com, Elcoirllord.com, Elcoriillord.com, Elforillord.com, 3lcorillord.com, Elcorollord.com, Elcorillordd.com, Elcorill0rd.com, Elcorillord/com, Elcoriolord.com, Ekcorillord.com, Elcorillord.xom, Elcorillord.co, Elcorilord.com, Elcoillord.com, 4lcorillord.com, Elcoriloord.com, Elcorillorr.com, Eldorillord.com, Elcorillord..com, Elcorrillord.com, http://www.elcorillord.com, El orillord.com, E.corillord.com, Elcorillord. om, Elcorilllrd.com, Elcorillrod.com, Elcoriklord.com, Elcorillorrd.com, Elcorillordcom, Elcorillkrd.com, Elcorillord.om, Elcorlilord.com, Elco4illord.com, Elcorillrd.com, Elcprillord.com, Elcodillord.com, Elcorillordc.om, Ecorillord.com, Elcoorillord.com, Slcorillord.com, Elcorillorx.com, Elcorill9rd.com, Elcorilpord.com, Elcorillord.cpm, Elcorillord.fom, Elco5illord.com, Elc9rillord.com, Elcorillird.com, Elcirillord.com, Elcorillord.clm, Elcorillors.com, Elcorillord.vom, Elcorullord.com, Elcorillord.comm, Elcorillord.c9m, Eocorillord.com, Elcorilloord.com, Elcourillourd.com, E,corillord.com, Elcorkllord.com, Elcorillodr.com, Elcorillodd.com, Elcorillord,com, Elcorillotd.com, Elcorillod.com, Elcorillor.dcom, Elcor9llord.com, Elc0rillord.com, Elcoril,ord.com, Elcorilllord.com, Elcorillo5d.com, EElcorillord.com, Elxorillord.com, E;corillord.com, Elcori;lord.com, Elcorllord.com, Elccorillord.com, Elcrillord.com, Elcoril.ord.com, Elcorilkord.com, Elcorillord.cmo, Ellcorillord.com, Eclorillord.com, Elcorillo4d.com, Rlcorillord.com, wwwelcorillord.com, Elclrillord.com, ElcorillordLcom, Dlcorillord.com, Elcorillofd.com, Elcorillore.com, Elcorillord;com, lcorillord.com, Elcorillorc.com, Elcorillord.c0m, Elcorillord.dom, Elcorillord.ocm."

Surely you have to see where many people would consider that either keyword stuffing, gibberish or typo spam.

Re: My history of (mostly failed) side projects and startups

#54
I like hearing about these and would be interested in hearing more, but can't imagine everyone would want HN flooded with personal failure histories. Is it worth cataloguing them somewhere, with potential for HNers to comment on or takeover or try again?

Call it something like 'Start Up Down'?

What do people think? Obviously not really something that's going to become a lucrative venture but it wouldn't be tough to create either.

Re: My history of (mostly failed) side projects and startups

#55
post #39

Earlier quoted context omitted.

Thanks for the links and your transparency. I will read up. Just to clarify - the reason I see this as relevant is because I'm a regular DDG user and put a lot of trust in your privacy claims (no logging of queries, etc).

Yup, I understand. If you're ever in Philadelphia, feel free to stop by and I can show you no logging in real time. As for NDB, the biggest points from my perspective are that: --you could opt-out from emails or remove yourself from the database at any time. --you could see a detailed explanation of how every aspect of the site worked before signing up, on a page I spent countless hours writing and tweaking. --simila…

I'm intrigued by the issues raised by selling a company which has personal data like in the case of NDB.

When you sold NDB did you have any concerns about how the new owners would treat members and their data? Knowing what has happened since (which--for some people at least--seems to be controversial) the sale would you do anything differently if you could?

A few years ago I had a company approach me to sell a small site I was running but I was never quite convinced they weren't just spammers/scammers wanting a customer list and felt like I owed my users more than that.

Re: My history of (mostly failed) side projects and startups

#56
post #13

Looks like you were startupping since 2000. Except NamesDatabase there haven't been any other exits. So how did you feed yourself until then? I am guessing your later projects were taken care by that exit.

Real job for one year. Then consulting. Then exit. Also, my wife has a job.

Thanks

Re: My history of (mostly failed) side projects and startups

#57

Earlier quoted context omitted.

Yup, I understand. If you're ever in Philadelphia, feel free to stop by and I can show you no logging in real time. As for NDB, the biggest points from my perspective are that: --you could opt-out from emails or remove yourself from the database at any time. --you could see a detailed explanation of how every aspect of the site worked before signing up, on a page I spent countless hours writing and tweaking. --simila…

I'm intrigued by the issues raised by selling a company which has personal data like in the case of NDB. When you sold NDB did you have any concerns about how the new owners would treat members and their data? Knowing what has happened since (which--for some people at least--seems to be controversial) the sale would you do anything differently if you could? A few years ago I had a company approach me to sell a small…

Definitely. The buyers were a public company who had recently bought classmates.com, were changing out the management, and had wanted to make it into a better site so-to-speak. I haven't substantiated any of the post claims, so I can't really speak to what happened. However, I would say that I think it really makes a difference if the team is going to stay or not.

Re: My history of (mostly failed) side projects and startups

#58

Earlier quoted context omitted.

Here's my (completely unsubstantiated) theory. It happend literally the day after crossing $500 & 50K views in adsense. I'm guessing one of those was a trigger for manual review by some contractor, perhaps overseas. They looked at my sites for 3 sec, found them to be cookie-cutter and decided to blacklist the account. I get the impression they shoot first, ask questions later. I didn't feel like dealing with it all o…

I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links. I mean, if you're auto-generating a page…

Thanks Matt. I was building a business off of these domains and realized that Google rankings were the biggest wildcard, and really didn't want any trouble. So I read the Webmaster guidelines closely and often and didn't think I was violating them.

However, I realize some were closer to the line and I should have focused on being less cookie-cutter and more useful in the domains that were really better (more farther along). I had always intended on coming back and working more on each, but wanted to get placeholders up quickly because it takes a while to get backlinks and indexed.

I guess I'm saying I had hoped I would have at least been contacted with a warning and what was found objectionable before just being totally blacklisted with no reason given. I would have also hoped that each site would have been addressed individually. If I had been contacted and you had said, hey, you need to remove these misspellings off of these sites, I would have done it immediately.

Here are some comments on the above though. Again, from my perspective these weren't violating the guidelines because the pages were useful from the user's perspective and there were no hidden tricks going on.

First off, there were actually many categories of sites, domains was just one of them. Others were sports stats, definitions, language, medical, and addresses. For each site I made, I was modelling it off of other sites that had gotten great Google rankings for years. I had hoped to eventually improve the UX on those sites and get similar rankings. For domains, I'm talking mainly about who.is and domaintools.com.

Each domain had a static site index, and that's what you linked to above in the screen shot. The extensive ones weren't really meant to be browsed, but just so search engines could find the pages (pre my knowledge of sitemaps). It's no different than any of the other static sitemaps, e.g. http://who.is/whois_index/index.php, and most of them looked better than the screenshot.

That one in particular came from the code for the streetsandzips site that was a big tag cloud. I was trying to find ways to make the static site better, and that was one of them. It looks better when the fonts are of different sizes :). I had intended for that site to make them different sizes based on the traffic numbers, so Google, Facebook, would be really big, etc. On the streetsandzips site the bigger cities are bigger.

In fact, I believe I evolved the sites so that those (site index) pages had noindex,follow on them such that they wouldn't come up on search results. I also added a search engine (Google custom search) on each page as well. I don't remember if I got to the tag cloud sizes for this particular domain at the time of blacklisting.

As for the misspellings, I did mess around with those, but not on all sites, and I believe at the time they were blacklisted that had been removed from most of the domains, if not all.

Common misspelling and typos as you know is a tool that people provide to those who buy domains. I built it for that purpose, and wanted to see how many people were actually searching for this stuff, so added it to some of the domains. Turns out, a lot of people do. I didn't just tack it on to the footer or cloak it or whatever; I put it in with a purpose that people ask for, e.g. common misspellings and typos.

Additionally from the users perspective, if they got to this page by typing in one of those misspellings, they were getting a big link to the official site at top and then more information about that site, e.g. siteadvisor rating, traffic, etc. So it was essentially functioning as one-click Did you mean x.

I'm happy to answer more questions about it. But it is pretty clear that it was still shoot first and ask questions later. No one ever contacted me about anything. I wasn't trying to hide anything from Google. It was all in my personal adsense account.

I can understand from a search engine perspective, banning sites. But given I already had a relationship with Google, I expected to be contacted. In fact, at one point I had a call with an Adsense guy from Google trying to help me better optimize my sites for Google! He looked at them and had no issues with them, so I thought I was fine.

Also, IIRC I submitted at least one re-inclusion request after being banned, and never heard a response back from that either. Before submitting that request I did a top to bottom review and tried to remove anything even close to the line, including misspellings I believe.

Re: My history of (mostly failed) side projects and startups

#59

Earlier quoted context omitted.

I don't think that was the issue. The fact is that if you've got dozens of websites, each of which has lists of domains/IPs like http://www.mattcutts.com/images/verypopularwebsites-com.png , that is the sort of thing that users complain about and don't want showing up when they do a search. Especially if sites have autogenerated boilerplate content for each one of those links. I mean, if you're auto-generating a page…

Thanks Matt. I was building a business off of these domains and realized that Google rankings were the biggest wildcard, and really didn't want any trouble. So I read the Webmaster guidelines closely and often and didn't think I was violating them. However, I realize some were closer to the line and I should have focused on being less cookie-cutter and more useful in the domains that were really better (more farther…

From what I've seen Google doesn't contact people :) My guess is they also have a policy of not sharing reasons for getting blacklisted, to ensure they're not giving spammers an easy way to fix their website.

They claim they respond to all "Site reconsideration" requests. I had to file one once, they did respond, but with a very non-informative and unhelpful response.

Re: My history of (mostly failed) side projects and startups

#60

Earlier quoted context omitted.

Thanks Matt. I was building a business off of these domains and realized that Google rankings were the biggest wildcard, and really didn't want any trouble. So I read the Webmaster guidelines closely and often and didn't think I was violating them. However, I realize some were closer to the line and I should have focused on being less cookie-cutter and more useful in the domains that were really better (more farther…

From what I've seen Google doesn't contact people :) My guess is they also have a policy of not sharing reasons for getting blacklisted, to ensure they're not giving spammers an easy way to fix their website. They claim they respond to all "Site reconsideration" requests. I had to file one once, they did respond, but with a very non-informative and unhelpful response.

Yeah, in retrospect I should have taken it slower and not gotten as close to the line in the first place. It's totally my fault, and I'm not bitter. As you can tell from the OP, I've had a lot of failure, and I similarly learned from this one.
Post reply on HN