Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Search without Google Tracking You (My 'pseudo' Startup)

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Search without Google Tracking You (My 'pseudo' Startup)

#11

Post by fadmmatt »

drm237 wrote: post #10
bls wrote: post #9

Instead of giving our private information to Google to use for 18 months, we are giving it to you to use indefinitely? Where is your privacy policy? And, what is your hosting provider's policy? I found out recently that my hosting provider refused to make any guarantees; that means that I cannot make any guarantees either, since my hosting provider has full access to my server.

A privacy policy is very high on my list of things to do, along with an explanation of how it works so that people can feel confident in their privacy. Right now, the only thing I record is a SHA1 has of everyone's IP address and a timestamp so that I can track how many unique people have used it. With the SHA1 hash, there's no (very difficult) way to trace that back to the user. I'm also talking to my host about the…

I hope that's SHA1 plus a secret salt.

Otherwise, I can just build a reverse map of SHA1'd ips.

for ip = 0 to 2^32: unhash[SHA1(ip)] = ip.

Re: Search without Google Tracking You (My 'pseudo' Startup)

#12

Post by drm237 »

fadmmatt wrote: post #11
drm237 wrote: post #10

Earlier quoted context omitted.

A privacy policy is very high on my list of things to do, along with an explanation of how it works so that people can feel confident in their privacy. Right now, the only thing I record is a SHA1 has of everyone's IP address and a timestamp so that I can track how many unique people have used it. With the SHA1 hash, there's no (very difficult) way to trace that back to the user. I'm also talking to my host about the…

I hope that's SHA1 plus a secret salt. Otherwise, I can just build a reverse map of SHA1'd ips. for ip = 0 to 2^32: unhash[SHA1(ip)] = ip.

Yes it is. But first you'd have to get all of the results from my database, which is cleared every 24 hours (this is not a challenge!).

Re: Search without Google Tracking You (My 'pseudo' Startup)

#13

Post by paulgb »

cedsav wrote: post #8
drm237 wrote: post #6

Earlier quoted context omitted.

The thing is, we're not making money from it in any way (which is why I call it a pseudo' startup), and because of that fair use can apply. Also, think about google. They scrape the entire web and profit by serving that as content...

Fair use certainly doesn't apply. You're using Google search technology (which btw, involves a bit more than 'scraping' the web) and stripping out the ads (their source of revenue). Expect a cease and desist letter soon.

Fair use doesn't apply, but if you can get your hands on an old API key it will still work. I built something similar (here: http://www.paulbutler.org/archives/endless-google-search/) and I just used an old Google API key, so I don't violate their TOS.

It makes me wonder though, since you can't make any money with it, how do you plan to pay for hosting fees? Surely you eventually intend to profit, what will you do then?

Re: Search without Google Tracking You (My 'pseudo' Startup)

#14

Post by drm237 »

paulgb wrote: post #13
cedsav wrote: post #8

Earlier quoted context omitted.

Fair use certainly doesn't apply. You're using Google search technology (which btw, involves a bit more than 'scraping' the web) and stripping out the ads (their source of revenue). Expect a cease and desist letter soon.

Fair use doesn't apply, but if you can get your hands on an old API key it will still work. I built something similar (here: http://www.paulbutler.org/archives/endless-google-search/ ) and I just used an old Google API key, so I don't violate their TOS. It makes me wonder though, since you can't make any money with it, how do you plan to pay for hosting fees? Surely you eventually intend to profit, what will you do t…

Correct me if I'm wrong, but isn't the API is limited to 1000 searches per day correct?

No, I don't plan on profiting from this. This is a very low impact system and I don't mind footing the bill for the hosting costs as long as it's of value to the community.

Re: Search without Google Tracking You (My 'pseudo' Startup)

#15

Post by jsjenkins168 »

drm237 wrote: post #10
bls wrote: post #9

Instead of giving our private information to Google to use for 18 months, we are giving it to you to use indefinitely? Where is your privacy policy? And, what is your hosting provider's policy? I found out recently that my hosting provider refused to make any guarantees; that means that I cannot make any guarantees either, since my hosting provider has full access to my server.

A privacy policy is very high on my list of things to do, along with an explanation of how it works so that people can feel confident in their privacy. Right now, the only thing I record is a SHA1 has of everyone's IP address and a timestamp so that I can track how many unique people have used it. With the SHA1 hash, there's no (very difficult) way to trace that back to the user. I'm also talking to my host about the…

Can you use a SHA-2 hash instead? Those variants are considerably more difficult to break.

A problem I see with your service compared to an anonymizing proxy like Tor is that you are still a single point of failure(please correct me if I'm wrong though). If you were legally forced to turn over search records (as the govt was attempting with google a while back), then the requests could be traced directly back to the user.

You mention clearing the database daily which is a good idea. But again if it was compromised and a snap shot could be taken, then a brute force crack of your SHA-1 hashes would be possible. Basically, everyone is trusting the security of your database. A misdirection service which telescopes the request through interconnected proxies will not have this single point of failure issue.

Not criticizing your implementation, just making some observations. I think this is a great idea. Mainly your site is so easy for people to use, not needing to install a client application.

Re: Search without Google Tracking You (My 'pseudo' Startup)

#16

Post by drm237 »

jsjenkins168 wrote: post #15
drm237 wrote: post #10

Earlier quoted context omitted.

A privacy policy is very high on my list of things to do, along with an explanation of how it works so that people can feel confident in their privacy. Right now, the only thing I record is a SHA1 has of everyone's IP address and a timestamp so that I can track how many unique people have used it. With the SHA1 hash, there's no (very difficult) way to trace that back to the user. I'm also talking to my host about the…

Can you use a SHA-2 hash instead? Those variants are considerably more difficult to break. A problem I see with your service compared to an anonymizing proxy like Tor is that you are still a single point of failure(please correct me if I'm wrong though). If you were legally forced to turn over search records (as the govt was attempting with google a while back), then the requests could be traced directly back to the…

Interesting points. So if someone was able to get my php code, they could find the salt, dump the database, generate the lookup tables by hashing every IP address possible plus the salt, and then they would be able to figure out every IP that has used the site since midnight. But, they would still only have your IP as I do not record any of the search results. This frame of events also shows that the hash algorithm I use really doesn't matter. It's protecting the salt and the database that matter most. Anyone have thoughts on that?

Also, another feature I was thinking of adding was an ssl option so you could securely access the site. However, as I don't make any money from the site, it becomes more difficult to justify additional expenses.

Re: Search without Google Tracking You (My 'pseudo' Startup)

#17

Post by jsjenkins168 »

drm237 wrote: post #16
jsjenkins168 wrote: post #15

Earlier quoted context omitted.

Can you use a SHA-2 hash instead? Those variants are considerably more difficult to break. A problem I see with your service compared to an anonymizing proxy like Tor is that you are still a single point of failure(please correct me if I'm wrong though). If you were legally forced to turn over search records (as the govt was attempting with google a while back), then the requests could be traced directly back to the…

Interesting points. So if someone was able to get my php code, they could find the salt, dump the database, generate the lookup tables by hashing every IP address possible plus the salt, and then they would be able to figure out every IP that has used the site since midnight. But, they would still only have your IP as I do not record any of the search results. This frame of events also shows that the hash algorithm I…

You mentioned that you keep timestamp info for each inbound connection, is that a requirement? I only ask b/c this could be used to match up a request on the search engines servers (with your servers IP as the source) with the connection on your server to pinpoint the user in the event your database was compromised.

One thing you could do which should be easy is send chaff. Randomly send out connection requests to some of search engines from your server even though a user is not requesting the data. It makes tying back connections to the users more difficult because you dont know which request is real and which is fake.

SSL would eventually be important because it would protect against man-in-the-middle attacks. Someone could hijack connections to your server claiming to be you and then get all of the requests. Users could potentially be putting in very sensitive information so this could be a big deal. There will also be protection from someone sniffing inbound requests that come into your server as the channel is encrypted.

I understand the expenses thing, so I wouldnt worry too much about that. I'd prefer your service be free and not use SSL than to charge for usage. Although I wouldnt mind some ads, you could monetize a bit on that if you wanted..

Re: Search without Google Tracking You (My 'pseudo' Startup)

#18

Post by drm237 »

jsjenkins168 wrote: post #17
drm237 wrote: post #16

Earlier quoted context omitted.

Interesting points. So if someone was able to get my php code, they could find the salt, dump the database, generate the lookup tables by hashing every IP address possible plus the salt, and then they would be able to figure out every IP that has used the site since midnight. But, they would still only have your IP as I do not record any of the search results. This frame of events also shows that the hash algorithm I…

You mentioned that you keep timestamp info for each inbound connection, is that a requirement? I only ask b/c this could be used to match up a request on the search engines servers (with your servers IP as the source) with the connection on your server to pinpoint the user in the event your database was compromised. One thing you could do which should be easy is send chaff. Randomly send out connection requests to so…

It's true that timestamps are probably no longer important. I initially had them in there to monitor usage while the test group was fairly limited. I will remove them in the next day or two as the site gets going (or dies...).

The idea of a chaff is interesting and It wouldn't be too difficult to implement. Thanks for the idea.

Re: Search without Google Tracking You (My 'pseudo' Startup)

#20

Post by paulgb »

drm237 wrote: post #14
paulgb wrote: post #13

Earlier quoted context omitted.

Fair use doesn't apply, but if you can get your hands on an old API key it will still work. I built something similar (here: http://www.paulbutler.org/archives/endless-google-search/ ) and I just used an old Google API key, so I don't violate their TOS. It makes me wonder though, since you can't make any money with it, how do you plan to pay for hosting fees? Surely you eventually intend to profit, what will you do t…

Correct me if I'm wrong, but isn't the API is limited to 1000 searches per day correct? No, I don't plan on profiting from this. This is a very low impact system and I don't mind footing the bill for the hosting costs as long as it's of value to the community.

I see, good work then.

You're right, I forgot about the API limit.

Post reply on HN