extract domain name from url

Sometimes I need to find domain name from url in my program for various purposes (most of the time in my crawlers). So far I used the following function that takes an url and returns the domain name:


def find_domain(url):
    pos = url[7:].find('/')
    if pos == -1:
        pos = url[7:].find('?')
        if pos == -1:
            return url[7:]
        url = url[7:(7+pos)]
        return url


But today I found a module named urlparse. So my function now looks like this:


def find_domain2(url):
    return urlparse(url)[1]

The new one is much better I think.

Check urlparse for details.

Comments

alexandre said…
This will return the hostname, not the domain name. Anyway, thanks for sharing.
April 16, 2008 at 1:27 AM
Unknown said…
Hey, i optimized your code:

from urlparse import urlparse
parsed = urlparse('http://example.com')
print parsed.hostname
July 20, 2010 at 9:08 PM
Swagat said…
exactly my point urlparse gives the hostname, not domain name
May 9, 2011 at 12:24 PM
Henri Salo said…
You should also try Python-module tldextract.
April 27, 2014 at 2:48 PM
Post a Comment

Popular posts from this blog

Strip HTML tags using Python

We often need to strip HTML tags from string (or HTML source). I usually do it using a simple regular expression in Python. Here is my function to strip HTML tags: def remove_html_tags(data): p = re.compile(r'<.*?>') return p.sub('', data) Here is another function to remove more than one consecutive white spaces: def remove_extra_spaces(data): p = re.compile(r'\s+') return p.sub(' ', data) Note that re module needs to be imported in order to use regular expression. Here you can find an updated code that gets the text from html: http://love-python.blogspot.com/2011/04/html-to-text-in-python.html
Read more

lambda magic to find prime numbers

Find prime number up to 100 with just 3 lines of code. The 4th line is for printing ;) nums = range(2, 100) for i in range(2, 10):     nums = filter(lambda x: x == i or x % i, nums) print nums Isn't it amazing? Check my new post about Prime number: http://love-python.blogspot.com/2010/11/prime-number-generator-in-python-using.html
Read more

Convert text to ASCII and ASCII to text - Python code

Python has a very simple way to convert text to ASCII and ASCII to text. To find the ASCII value of a character use the ord() function. Example: >>> ord('B') 66 To get a character from it's ASCII value use the chr() function. Example: >>> chr(65) 'A' Here is a program to get the list of all characters along with their values: # program to print the list of characters and their ASCII values for value in range(0, 255): print "ASCII Value:", value, "\t", "Character:", chr(value), "\n" Here is another simple ASCII encoding-decoding program: # Convertion from text to ASCII codes message = raw_input("Enter message to encode: ") print "Decoded string (in ASCII):" for ch in message: print ord(ch), print "\n\n" # Convertion from ASCII codes to text message = raw_input("Enter ASCII codes: ") decodedMessage = "" for item in message.split(): decodedMessage += chr
Read more