The Craigwatch Story
craigwatch.com
The Craigwatch Story
1–10 of 50 posts
Re: The Craigwatch Story
#2In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates.
In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader?
Did Craigwatch use RSS feeds, or was it scraping material from the rendered HTML?
Re: The Craigwatch Story
#3In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…
I do that too, but it can be pretty late. At least hours after the item is posted. Which isn't bad for some things, but sometimes you _really_ want to know. :D
I've never actually heard of Craigswatch, but I think it's something most engineers have done at least a couple of times. I've got a script I use to watch the FredMiranda forums any time I need some new camera gear, for example.
Re: The Craigwatch Story
#4In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…
Looks like it was using XPath to parse the HTML: https://github.com/beaulm/craigwatch/blob/master/post.php
I imagine the 'selling' point was users not even having to know what RSS even means, let alone how to script it.
Re: The Craigwatch Story
#5I would think CL using a proxy that silently blocks client ips that interact with the service suspiciously may be cheaper than sending out lawyer letters and more immediately effective. Just prefer technical solutions to legal threatening I guess..
Re: The Craigwatch Story
#6I used Craigwatch. It was a useful tool - when a search pops, I get an email. I used to use Google Reader for this but when Reader died so did my RSS reading habit, so now email is the only part of my daily workflow that I poll frequently.
The real mystery to me is why Craigslist shut the service down, because (protocols aside) it wasn't fundamentally different from any other way of consuming craigslist. Craigslist's terms of service is actually pretty confusing:
USE. You agree not to use or provide software (except for general purpose web browsers and email clients, or software expressly licensed by us) or services that interact or interoperate with CL, e.g. for downloading, uploading, posting, flagging, emailing, search, or mobile use. Robots, spiders, scripts, scrapers, crawlers, etc. are prohibited, as are misleading, unsolicited, unlawful, and/or spam postings/email. You agree not to collect users’ personal and/or contact information (“PI”).
...which if you think about it, bans the use of RSS readers too. There's a short blurb on the RSS page:
craigslist RSS feeds are for your personal use only, and are not available for commercial use without first obtaining a license from craigslist. Please consult our for more information on using craigslist RSS feeds.
The only relevant part of the Terms of Use is the section mentioned above, which doesn't really shed any light on the situation.
Honestly this is really confusing. Once again, Craigslist disappoints.
Re: The Craigwatch Story
#7In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…
Useful free stuff is typically gone in minutes. Only junk remains posted.
Its RSS feeds have a refresh period of 1 hour, which is simply too long to have the opportunity to grab hot items.
As Craigslist has very shallow category listings in its "free" category, where you can't separate furniture from clothes as an example, the site forces users to rely on manually refreshing pages, which change very often.
I have better user experience using Craigslist from my iPad. Craigslist appears to selectively make the service available to app makers but not to websites.
In the past 3 days of shopping for a specific car within about 15 Craigslist local communities, I managed to get automatically blocked by the service several times. What I am looking for is very specific and I don't mind traveling a bit to get what I want.
I ended up having to write code with sleep timers to reduce my number of web queries once I narrowed what I wanted.
An essential feature in the iPad app is that I can cross off listings that are no longer interesting to me and highlight favorites. There is no listing cross-off feature for the website.
I guess my next step is to add that feature. laugh
Re: The Craigwatch Story
#8A while back, I hacked together notifications using RSS, but lately, I just use IFTTT...
Re: The Craigwatch Story
#9Did craigwatch.com agree to the terms on the site?
Re: The Craigwatch Story
#10A while back, I hacked together notifications using RSS, but lately, I just use IFTTT... https://ifttt.com/craigslist
I use IFTTT's craigslist channel all the time for long-term polling of craigslist. Not sure how instant it is, but notifications are pretty reliable.