check status proxy address

Often we need to use proxy addresses in our web spiders / crawler. But most of the time the proxies don't work. So I made a little python program to test the proxy IPs. Let's look into the code:

import urllib2, socket

socket.setdefaulttimeout(180)

# read the list of proxy IPs in proxyList
proxyList = ['125.76.226.9:80', '213.55.87.162:6588'] # there are two sample proxy ip

for item in proxyList:
if is_bad_proxy(item):
print "Bad Proxy", item
else
print item, "is working"


def is_bad_proxy(pip):
try:
proxy_handler = urllib2.ProxyHandler({'http': pip})
opener = urllib2.build_opener(proxy_handler)
opener.addheaders = [('User-agent', 'Mozilla/5.0')]
urllib2.install_opener(opener)
req=urllib2.Request('http://www.your-domain.com') # change the url address here
sock=urllib2.urlopen(req)
except urllib2.HTTPError, e:
print 'Error code: ', e.code
return e.code
except Exception, detail:

print "ERROR:", detail
return 1
return 0


Hope the proxy checker will be useful to you!
You can use this list to test your script.

Comments

ronin1770 said…
any python based web crawler - you can suggest???
Tamim Shahriar said…
I have heard that 'harvester' is good. I didn't explore it though.
Mark Stone said…
This comment has been removed by the author.
nikhiljjoshi said…
hey nice post

i was trying to open a webpage with the good proxy but somehow could not manage to do so

i was using the webbrowser function

can u help me pls??
Ant said…
I was looking for a good proxy checker in python. This could easily be edited to run through a file!

Sweet dude, on my blog i post proxys lists :D
Anonymous said…
Man! I love you!!
Thanks a lot for this cool code!
Thanks!!!!!
Unknown said…
Your source enabled me to create a great proxy tester. Thanks.

http://sourceforge.net/apps/mediawiki/proxytest/index.php?title=Main_Page
GamesBook said…
What I need is way to extract the default set of proxies that are already installed/available on a specific machine (which I can then test with your code)... any ideas on that?

Popular posts from this blog

lambda magic to find prime numbers

Convert text to ASCII and ASCII to text - Python code

Adjacency Matrix (Graph) in Python