Posts

Showing posts with the label python unicode

UnicodeDecodeError in CSV writer in Python

I faced another error with UnicodeWriter class. Now it's UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 21: ordinal not in range(128) . More details: Traceback (most recent call last):   File "cpx_parser.py", line 284, in     main()   File "cpx_parser.py", line 278, in main     writer.writerow(csv_li)   File "cpx_parser.py", line 29, in writerow     self.writer.writerow([s.encode("utf-8") for s in row]) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 21: ordinal not in range(128) So, I had to decode s using utf-8, then convert it into unicode. And that solved the problem. Here is what I did: self.writer.writerow([unicode(s.decode("utf-8")).encode("utf-8") for s in row]) To know more about unicode, you can check this link .

Python UNICODE encode / decode error

Today I was trying to scrape a Spanish site and got into trouble with some Spanish characters. I had to parse some messages from that Spanish website and post into twitter using my Python script. But for some reasons Spanish characters didn't show up in twitter status updates. I was in some trouble with the following error messages: UnicodeDecodeError: 'utf8' codec can't decode bytes in position 5778-5781: invalid data UnicodeEncodeError: 'ascii' codec can't encode character u'\xf3' in position 5778: ordinal not in range(128) Then I did Googling for some time and got this very useful link . I used this code to get the content of the website: import codecs import urllib2 url = '' # put the URL here usock = urllib2.urlopen(url) Reader = codecs.getreader("latin_1") fh = Reader(usock) data = fh.read() fh.close() usock.close() data = data.encode("latin_1") Though I first used utf-8 encoding rather than latin_1, but when I got t...