Live data from Hacker News

What technologies are under your site

underthesite.com

31–40 of 97 posts

Re: What technologies are under your site

#31

Interesting, but seems a little primitive compared to the cleverness in https://github.com/mitsuhiko/probe

Does probe try and fetch signal urls? (Like admin pages.). UnderTheSite.com makes a point of only looking at data that would normally be fetched by your browser. Probing a site for urls could be considered offensive.

Yes, it does probe at site URLs, you can look at libprobe.py to see all the indicators it takes into account.

Re: What technologies are under your site

#32

Hey guys, I made this site and just gave a talk about it at SHDH. Someone must have submitted it. Thanks for all your feedback, I really appreciate it!

Awesome! Suggestions:

Do a reverse lookup on the site's IP and add matchers for the hostnames that show up.

Mention if they're round robining their DNS.

Add nmap OS fingerprinting.

Do a traceroute and log the IP of the closest router (final hop) to the site and add matching for that.

Add a wiki interface and build a crunchbase like app.

Add archiving of data and monitoring over time (as netcraft did in their original app).

Re: What technologies are under your site

#33

Hey guys, I made this site and just gave a talk about it at SHDH. Someone must have submitted it. Thanks for all your feedback, I really appreciate it!

Awesome! Suggestions: Do a reverse lookup on the site's IP and add matchers for the hostnames that show up. Mention if they're round robining their DNS. Add nmap OS fingerprinting. Do a traceroute and log the IP of the closest router (final hop) to the site and add matching for that. Add a wiki interface and build a crunchbase like app. Add archiving of data and monitoring over time (as netcraft did in their original…

nmap OS fingerprinting seems a bit aggressive, but reverse-DNS checking seems like a great way to check which hosting platform a site uses.

Re: What technologies are under your site

#40

This doesn't really tell you anything about a site. Most of the interesting stuff happens on the server side.

That's getting less and less true with the amount of stuff happening in Javascript.

Underthesite itself does its work behind a server:

http://underthesite.com/sites/underthesite.com

All you can tell from it querying itself is that the work happens somewhere behind rails.

Post reply on HN