Show HN: Groupon Scraper in Python

#1

Post by svrocks » Fri, Jun 03, 2011, 5:20 PM UTC

https://github.com/jxchen/DailyDealScrapers

On the Groupon IPO thread someone mentioned people selling scraped groupon deal data. To satisfy my curiosity I made my own. Pretty interesting to play around with the data and see how much money their deals are making.

Re: Show HN: Groupon Scraper in Python

#2

Post by jparicka » Fri, Jun 03, 2011, 6:25 PM UTC

Neat!

Re: Show HN: Groupon Scraper in Python

#3

Post by drats » Fri, Jun 03, 2011, 8:12 PM UTC

Looks great. I'd comment the code though, and look into a real html parser.

Re: Show HN: Groupon Scraper in Python

#4

Post by ayb » Fri, Jun 03, 2011, 8:29 PM UTC

I'm pretty sure you can get that same data via Groupons own RSS feed, no?

Such as: http://feeds.feedburner.com/grouponnew-york

Re: Show HN: Groupon Scraper in Python

#5

Post by svrocks » Fri, Jun 03, 2011, 10:48 PM UTC

ayb wrote: post #4

I'm pretty sure you can get that same data via Groupons own RSS feed, no? Such as: http://feeds.feedburner.com/grouponnew-york

Yes but I wanted to be able to collect sale statistics, so either way some scraping would need to be done on the links

Re: Show HN: Groupon Scraper in Python

#6

Post by svrocks » Fri, Jun 03, 2011, 10:49 PM UTC

drats wrote: post #3

Looks great. I'd comment the code though, and look into a real html parser.

Ha yeah the code is a mess. I have used BeautifulSoup before but I don't understand how it works well enough and just wanted to get something out the door

Re: Show HN: Groupon Scraper in Python

#7

Post by demirhan » Sat, Jun 04, 2011, 8:10 AM UTC

I was looking for their revenues for april and may. Their revenues were declined for previous months.

Business Insider didnt report their revenue for april and may. Thus, did you have a chance to scape their past data for may and april ?

Re: Show HN: Groupon Scraper in Python

#8

Post by svrocks » Sat, Jun 04, 2011, 1:46 PM UTC

demirhan wrote: post #7

I was looking for their revenues for april and may. Their revenues were declined for previous months. Business Insider didnt report their revenue for april and may. Thus, did you have a chance to scape their past data for may and april ?

no i just made this yesterday, it only takes snapshots of current deals. If you want to collect revenues going forward I would set up some kind of cron job and run it every night at a reasonable time (11pm Eastern US time would probably work, but you'd underestimate on the West Coast deals)

Re: Show HN: Groupon Scraper in Python

#9

Post by dazzla » Sat, Jun 04, 2011, 6:19 PM UTC

You can't get what you need from their API? http://www.groupon.com/pages/api