Create child process in Python - Subprocess module

Recently in one of my projects I had to create subprocess (child process) in Python. Actually I had to run multiple instances and didn't want to wait till those processes finish their job. I used the subprocess module to create those process. I am not sharing my actual code rather giving some sample code here that I wrote to understand the use of subprocess module myself which show you how to create child process (for which parent process need not wait for all the child processes to finish execution) using subprocess module.

1. create a file named test.py

#!/usr/bin/env python

import os, subprocess

print "hello"

urlList = ["http://python.org",
"http://muktosoft.com",
"http://bdosn.org",
"http://matholympiad.org.bd",
"http://bbc.co.uk",
"http://cricinfo.com", 
"http://bdpy.org"]


for url in urlList:
    subprocess.Popen(["python", "test2.py", url])

print "good bye\n"

2. now create another file in the same directory named test2.py.

import urllib2
import sys
from urlparse import urlparse

url = sys.argv[1]

print "\nURL: ", url

usock = urllib2.urlopen(url)
data = usock.read()
usock.close()

filename = urlparse(url).netloc + ".html"
fp = open(filename, "w")
fp.write(data)
fp.close()

Now run the program test.py.

test.py program doesn't wait for all the child processes to be finished. It just starts those and terminated. Each child process is terminated when they finish execution. Don't forget to notice the order in which html files are created in your directory. :)

You might need to make the test2.py executable (chmod +x test2.py) but I am not sure whether it was necessary.