Live data from Hacker News

The Craigwatch Story

craigwatch.com

11–20 of 50 posts

Re: The Craigwatch Story

#13
post #7

In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…

Useful free stuff is typically gone in minutes. Only junk remains posted. Its RSS feeds have a refresh period of 1 hour, which is simply too long to have the opportunity to grab hot items. As Craigslist has very shallow category listings in its "free" category, where you can't separate furniture from clothes as an example, the site forces users to rely on manually refreshing pages, which change very often. I have bet…

Wiseleo, is that really true about the refresh rate? RSS readers have a configurable rate. Often there is a one hour default in order to be reasonably nice to the server. Basically, a Craigslist RSS item is just a URL with the search parameters embedded. Maybe I'm wrong, but I suspect that whenever you fetch this URL, the server executes the search and produces the results as RSS XML items, so the fetch rate is controlled by you (your reader). Or are you saying there is some additional throttling on the server side, so that RSS-based searches do not see up-to-the-second updates that are visible through the Web interface? So that no matter how often you refresh the RSS feed, you don't see new items that are already visible via HTML?

Re: The Craigwatch Story

#14

In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…

Looks like it was using XPath to parse the HTML: https://github.com/beaulm/craigwatch/blob/master/post.php I imagine the 'selling' point was users not even having to know what RSS even means, let alone how to script it.

Thanks for the link. Time to try out Google app engine's PHP preview.

Re: The Craigwatch Story

#15
post #9

Did craigwatch.com agree to the terms on the site?

There's probably a provision in the TOS that says that you implicitly agree to the terms by virtue of using the service. I don't know for sure, just speculating.

Re: The Craigwatch Story

#17

I used Craigwatch. It was a useful tool - when a search pops, I get an email. I used to use Google Reader for this but when Reader died so did my RSS reading habit, so now email is the only part of my daily workflow that I poll frequently. The real mystery to me is why Craigslist shut the service down, because (protocols aside) it wasn't fundamentally different from any other way of consuming craigslist. Craigslist's…

Craigslist has a really bad history of abusing its users. For instance, a few years back they changed the TOS so that CL becomes the copyright holder of all ads placed on the site.

Re: The Craigwatch Story

#18
A little bit off topic but interesting nonetheless.

I've known Beau for many years now and his writing style has improved so much. He is a testament to the "practice makes perfect" (or better at least) mantra. It's really interesting to see the transition in his travel blog from when he started writing to now. If you're interested you can check out. Compare the first post to the most recent. http://dangertravels.com/

Re: The Craigwatch Story

#19
post #7

In the past, I have sometimes used Craigslist's RSS feature to watch for the appearance of items. That is to say, the ability of CL to export a keyword search as an RSS feed that you can save in your feed reader and watch for updates. In what ways did Craigwatch improve over RSS? If it still operated today, why would I use that instead of scraping RSS items directly from Craigslist to my feed reader? Did Craigwatch u…

Useful free stuff is typically gone in minutes. Only junk remains posted. Its RSS feeds have a refresh period of 1 hour, which is simply too long to have the opportunity to grab hot items. As Craigslist has very shallow category listings in its "free" category, where you can't separate furniture from clothes as an example, the site forces users to rely on manually refreshing pages, which change very often. I have bet…

I used to give away good free stuff, but it's just not woth the hassle(going to the wrong people, people not showing up, the spam, over agressive Flaggers--you people need to get a life, etc.)

While I'm here. What's with the begging to lower an already low price on a service, or item. I used to price an item realistically, then knock off 40 percent. People still want to Haggle over the price. I now just double the price I think it's worth.

There's a huge need for another Craigslist. The site has gotten too Ghetto.

Re: The Craigwatch Story

#20
post #17

I used Craigwatch. It was a useful tool - when a search pops, I get an email. I used to use Google Reader for this but when Reader died so did my RSS reading habit, so now email is the only part of my daily workflow that I poll frequently. The real mystery to me is why Craigslist shut the service down, because (protocols aside) it wasn't fundamentally different from any other way of consuming craigslist. Craigslist's…

Craigslist has a really bad history of abusing its users. For instance, a few years back they changed the TOS so that CL becomes the copyright holder of all ads placed on the site.

So trying to fend off padmapper (whether one agrees with that or not) has suddenly become "a really bad history of abusing its users"? I guess scrapers are users.
Post reply on HN