Live data from Hacker News

Finding the best ticket price – Simple web scraping with Python

danielforsyth.me

11–20 of 36 posts

Re: Finding the best ticket price – Simple web scraping with Python

#12

Doesn't work for me. Which Python version is required? Traceback (most recent call last): File "./tickets.py", line 20, in for listing in soup.findall('p', {'class': 'row'}): TypeError: 'NoneType' object is not callable

I am using 2.7.6

Re: Finding the best ticket price – Simple web scraping with Python

#15

Doesn't work for me. Which Python version is required? Traceback (most recent call last): File "./tickets.py", line 20, in for listing in soup.findall('p', {'class': 'row'}): TypeError: 'NoneType' object is not callable

I am using 2.7.6

Maybe wrong BS version?

    $ sudo pip install BeautifulSoup
    Downloading/unpacking BeautifulSoup
      Downloading BeautifulSoup-3.2.1.tar.gz
      Running setup.py (path:/tmp/pip_build_root/BeautifulSoup/setup.py) egg_info for package BeautifulSoup
        
    Installing collected packages: BeautifulSoup
      Running setup.py install for BeautifulSoup
        
    Successfully installed BeautifulSoup
    Cleaning up...

Re: Finding the best ticket price – Simple web scraping with Python

#16

Earlier quoted context omitted.

That really isn't a good name.

Seems to be their thing QHandJoob QFlatBoob

Wow. Spurred on by your discoveries, I found this: http://weboob.org/applications/qhavedate

"QHaveDate is a graphical application able to interact with dating websites, and help you manage your numerous conquests."

I, uh, don't really know where to start with that.

Re: Finding the best ticket price – Simple web scraping with Python

#17

Earlier quoted context omitted.

I am using 2.7.6

Maybe wrong BS version? $ sudo pip install BeautifulSoup Downloading/unpacking BeautifulSoup Downloading BeautifulSoup-3.2.1.tar.gz Running setup.py (path:/tmp/pip_build_root/BeautifulSoup/setup.py) egg_info for package BeautifulSoup Installing collected packages: BeautifulSoup Running setup.py install for BeautifulSoup Successfully installed BeautifulSoup Cleaning up...

Ah yes thats the problem, I am using beautifulsoup4==4.3.2.

Try pip install beautifulsoup4

Re: Finding the best ticket price – Simple web scraping with Python

#18

Earlier quoted context omitted.

Maybe wrong BS version? $ sudo pip install BeautifulSoup Downloading/unpacking BeautifulSoup Downloading BeautifulSoup-3.2.1.tar.gz Running setup.py (path:/tmp/pip_build_root/BeautifulSoup/setup.py) egg_info for package BeautifulSoup Installing collected packages: BeautifulSoup Running setup.py install for BeautifulSoup Successfully installed BeautifulSoup Cleaning up...

Ah yes thats the problem, I am using beautifulsoup4==4.3.2. Try pip install beautifulsoup4

I found the problem. I think your listing ate '_'. It should be 'soup.find_all' instead of 'soup.findall' and 'link_end' instead of 'linkend'

Re: Finding the best ticket price – Simple web scraping with Python

#19
A shorter, more comprehensible version:

import requests

from bs4 import BeautifulSoup

from urlparse import urljoin

URL = 'http://philadelphia.craigslist.org/search/sss?sort=date&quer...

BASE = 'http://philadelphia.craigslist.org/cpg/'

response = requests.get(URL)

soup = BeautifulSoup(response.content)

for listing in soup.find_all('p',{'class':'row'}):

    if listing.find('span',{'class':'price'}):

        price = int(listing.text[2:6])

        if 100 

Re: Finding the best ticket price – Simple web scraping with Python

#20
post #19

A shorter, more comprehensible version: import requests from bs4 import BeautifulSoup from urlparse import urljoin URL = ' http://philadelphia.craigslist.org/search/sss?sort=date&quer... BASE = ' http://philadelphia.craigslist.org/cpg/' response = requests.get(URL) soup = BeautifulSoup(response.content) for listing in soup.find_all('p',{'class':'row'}): if listing.find('span',{'class':'price'}): price = int(listing.t…

Thanks for posting this, I am still very new to python and your website has taught me a lot. Appreciate the feedback.
Post reply on HN