Viewing profile — wrath

When websites get a ton of traffic the concern is that the algorithm that find fake data will not be accurate and start blocking paying customers. So it's a fine line between block…

I can't provide details on any innovations we've done with sites like google, but in general if you want to crawl google you'll want to get "many, many" IP addresses. I've heard of…

Competitive intelligence. Crawling thousands of website, mashing up the data to analyze competitiveness between them, and selling it back. For example, cost of flights. Different w…

I don't think I'm being hypocritical. I have no issues if people crawl my site, I even identify who they are and give them access to a private API. I do not generate any income fro…

I've built crawlers that retrieve billions of web pages every month. We had a whole team working modifying the crawlers to resolve website changes, to reverse engineer ajax request…

360pi, Ottawa Ontario, Canada Company Overview: 360pi helps top retailers compete and win in an era when shoppers expect and demand complete price transparency. Our customer base o…

I think it's that's a very good rate of growth for a new service. The scaling challenges alone are not obvious and slowly taking up market share is the safer thing to do. If I was …

I think @nunobrito covered most of what I wanted to say. Without knowing your product it's hard to give you the best advice, but here are things that worked for me in the past: * U…

I would politely insist that an email be sent out by the founder about your departure before the end of day. That way you get a chance of talking to your co-workers and explain the…

As far as I know, shader.io sells individual proxies. Luminati and Nohodo are exit nodes priced by the amount of bandwidth that you use.

Sorry, my bad... it's shader.io

If these guys are serious about their business they are using proxies to obfuscate who they are. There are many services now that are claiming to have a ton of IP addresses Luminat…

360pi Full-time onsite in Ottawa, Canada. http://360pi.com 360pi is a company organizing the worlds product data for retailers, brands and consumers. World class dev team, leading …

I love and miss RegexBuddy! If only they had a Mac/Linux version :( I've had thoughts about switching back to Windows while developing complex regular expressions.

I've built several businesses that either relied in-part to scrapping/indexing websites or solely relied on scrapping/index websites. I We never achieved the success of Google but …

I apologize if I offended anyone and/or if I said anything improper; it was not my intention. I wrote this quickly during the March "Who's hiring" thread and copy/pasted it today. …

360pi, Ottawa Ontario, Canada Company Overview: 360pi helps top retailers compete and win in an era when shoppers expect and demand complete price transparency. Our customer base o…

360pi, Ottawa Ontario, Canada Company Overview: 360pi helps top retailers compete and win in an era when shoppers expect and demand complete price transparency. Our customer base o…

1. You can try using bi-grams or even tri-grams to make you word list a little more precise. 2. Create a validation by manually identifying each review as positive or negative. Eac…

Amazon has made it much harder to scrape their site in recent months. They now give captchas to bots that try and access their site to often by the same IP address. Unless you have…

http://www.blogcdn.com/travel.aol.co.uk/media/2013/01/323ng1... My understanding is that this image was captured by a camera trigger that was setup National Geographic photographer…

I'm curious to know how developers and IT interact within other companies. I'm a CTO of a company that has 18 developers and 2 IT. One of my IT guys has decided to pursue other int…

A company that I started in the early 2000s used cold calling to generate leads. It was very affective to generate solid qualified leads. We were lucky that we go an experience col…

Good article but I would look at this "failure" from a glass half full perspective. You "won" because you've learned valuable lessons you can take to your next idea. I've had many …

This is a phishing attack waiting to happen! I never worked at a bank but I'm assuming (maybe I shouldn't) that there are a few people working there that know a thing or two about …