I need a script that will scrape [login to view URL] and save the information in an xml file. Only the United States "Free" section needs to be scraped and I need to ability to specify the US city (e.g. chicago), area (e.g. sox) and neighborhood (neighborhood should be optional since most cities don't offer this; see "sf bayarea", choose "sby", click "free" and notice the neighborhood dropdown). If a neighborhood is not specified, the script should parse the listings and pull the neighborhood listed next to the title (ignore postings without neighborhoods).
Here's the list of information that is needed from the scrape:
URL of Post
URL(s) of photos (if available)
Title of Post
"Reply to" Email Address
Post Description
Date Posted
City
Neighborhood
This script can be coded in Python (preferred) or Ruby
Here are a couple of example URLs:
[login to view URL] (no neighborhood specified in URL)
[login to view URL];neighborhood=109 (neighborhood specified in the URL)