Live data from Hacker News

Show HN: Crawling car dealerships for a real time consumer search engine

demanjo.com

31–40 of 66 posts

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#33

I'm seeing a fair few duplicates in the results, probably need to work on the algorithm for filtering these out.

Noticed!! some dealerships are operating under several distinct domain names which create these duplicates. We are currently working on a solution for this.

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#34
post #29

Just a weird error. If I accept the location tracking and use this url: http://www.demanjo.com/new/search?3=3670447604&0=3526732... Then it will show the search, but go to another search afterwards.

Hi, can your provide more details into this error? What location are you in?

Works perfectly for me from my location.

Will be very helpful.

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#35
post #31

How come there is no cars in San Francisco ? Yu should may be think of parsing craiglist ?

Our crawler is currently in motion and i see about 239 cars in SF. We are solely focusing on dealership websites.

The only suggestions I have when I type "San Fra" are : San Clara, MB San Josef Bay, BC San Josef, BC San Joseph Bay, BC"

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#36
post #13

Earlier quoted context omitted.

I really wish people would test their sites before doing this. I can understand if someone else submits your stuff and you had no idea it was going to happen (but even then...), but if you create an account with no history for the explicit purpose of submitting to HN as the OP did, you should at least test it under some semblance of load.

I would love to see a post on how to test your site for load issues like ones you would face from hn

I posted an Ask HN a few months ago asking what folks do to test but the best answer I got was "load test your website" :P

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#37
post #13

Earlier quoted context omitted.

I really wish people would test their sites before doing this. I can understand if someone else submits your stuff and you had no idea it was going to happen (but even then...), but if you create an account with no history for the explicit purpose of submitting to HN as the OP did, you should at least test it under some semblance of load.

I understand what you mean. but we have actually tested the site several times before posting it on HN. We are working hard at this and will be back shortly. Thanks.

Looks like everything's working now. Can I ask why when searching for cars in the US all the mileages are in Km?

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#38
post #35

Earlier quoted context omitted.

Our crawler is currently in motion and i see about 239 cars in SF. We are solely focusing on dealership websites.

The only suggestions I have when I type "San Fra" are : San Clara, MB San Josef Bay, BC San Josef, BC San Joseph Bay, BC"

Your IP address resolves to Canada.

You can only search for cars in your country.

Will have to change this if required at some point

Re: Show HN: Crawling car dealerships for a real time consumer search engine

#39
post #37

Earlier quoted context omitted.

I understand what you mean. but we have actually tested the site several times before posting it on HN. We are working hard at this and will be back shortly. Thanks.

Looks like everything's working now. Can I ask why when searching for cars in the US all the mileages are in Km?

True, will have to change this.
Post reply on HN