get content (html source) of an URL by HTTP POST method in Python
To retrieve or get content (html source) of an URL, sometimes we need to POST some values. Here I show you a sample python code that uses POST. import urllib, urllib2, time url = 'http://www.example.com' # write ur URL here values = {'key1' : 'value1', #write ur specific key/value pair 'key2' : 'value2', 'key3' : 'value3', } try: data = urllib.urlencode(values) req = urllib2.Request(url, data) response = urllib2.urlopen(req) the_page = response.read() print the_page except Exception, detail: print "Err ", detail Hope it will be useful for you while writing web crawler/spider, specially where values must be submitted using HTTP POST method to get or extract content from an URL. Please write your comments.