Issue 73: UTF decode problem
Status:  Fixed
Owner:
Closed:  Jan 2008
Reported by jadest...@gmail.com, Dec 15, 2007
 Posted by guest at 2007-07-09 13:08:29

Message from server

ICQ connection lost! Reason: [Failure instance: Traceback (failure with no
frames): exceptions.UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in
position 31: unexpected code byte
]

Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/twisted/python/log.py", line
53, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib/python2.4/site-packages/twisted/python/log.py", line
38, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib/python2.4/site-packages/twisted/python/context.py",
line 59, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib/python2.4/site-packages/twisted/python/context.py",
line 37, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File
"/usr/lib/python2.4/site-packages/twisted/internet/pollreactor.py",
line 164, in _doReadOrWrite
    why = selectable.doRead()
  File "/usr/lib/python2.4/site-packages/twisted/internet/tcp.py",
line 348, in doRead
    return self.protocol.dataReceived(data)
  File "/usr/local/pyicq-t-0.8a/src/tlib/oscar.py", line 647, in
dataReceived
    state=func(flap)
  File "/usr/local/pyicq-t-0.8a/src/tlib/oscar.py", line 764, in
oscar_Data
    func(snac)
  File "/usr/local/pyicq-t-0.8a/src/tlib/oscar.py", line 1222, in
oscar_03_0B
    self.updateBuddy(self.parseUser(snac[5]))
  File "/usr/local/pyicq-t-0.8a/src/legacy/icqt.py", line 179, in
updateBuddy
    status = status.encode("utf-8", "replace")
exceptions.UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position
31: unexpected code byte

Posted by guest at 2007-07-09 14:03:25

today i received same errors too=(

Posted by guest at 2007-07-09 14:12:11

I made a little patch that supresses failure due to this exception.
Now it works, but I'm not sure this is enough.


Before (legacy/icqt.py line 179):
--------
status = status.encode("utf-8", "replace")
--------

After:
--------
try:
 status = status.encode("utf-8", "replace")
except UnicodeError:
 pass
--------

//zlobnyme

Posted by guest at 2007-07-10 06:34:44

This patch dont 100% work correctly.

Jadestorm?

Posted by guest at 2007-07-10 07:05:14

on my server 100% work :-)

/icq.gelf.no-ip.org

Posted by Kepi at 2007-07-10 08:43:43

Nice solution.. this problem appears today on our servers and we were unable to
use icq transport at all.. Thank you guest :) for help

Posted by Kepi at 2007-07-10 08:46:33

Nice solution.. this problem appears today on our servers and we were unable to
use icq transport at all.. Thank you guest :) for help

Posted by guest at 2007-07-26 13:45:41

I found this "solution" will show some users who are actually away as
offline. Been working for some hours now to get a better solution. I can't
garantee the functionality of the code - however it does show some more people
online who seemed to be offline. You will find, that the away-message of those
people will be "Error while encoding status." and the status set to
away. I found this the best solution to handle the bogus people :-)

kind regards

konqi

ORIGINAL (line 179+):
		status = status.encode("utf-8", "replace")
		if user.flags.count("away"):
			self.getAway(user.name).addCallback(self.sendAwayPresence, user)
		else:
			c.updatePresence(show=show, status=status, ptype=ptype, url=url)
			self.oscarcon.legacyList.updateSSIContact(user.name, presence=ptype,
show=show, status=status, ipaddr=user.icqIPaddy, lanipaddr=user.icqLANIPaddy,
lanipport=user.icqLANIPport, icqprotocol=user.icqProtocolVersion, url=url)

MY VERSION:
 try:
                        try:
                                status = status.encode("utf-8",
"replace")
                        except UnicodeError:
                                raise
                        if user.flags.count("away"):
                               
self.getAway(user.name).addCallback(self.sendAwayPresence, user)
                        else:
                                c.updatePresence(show=show, status=status,
ptype=ptype, url=url)
                               
self.oscarcon.legacyList.updateSSIContact(user.name, presence=ptype, show=show,
status=status, ipaddr=user.icqIPaddy, lanipaddr=user.icqLANIPaddy,
lanipport=user.icqLANIPport, icqprotocol=user.icqProtocolVersion, url=url)
                except UnicodeError:
                        status = "Error while encoding status."
                        status = status.encode("utf-8",
"replace")
                        c.updatePresence(show='away', status=status,
ptype=ptype, url=url)
                        self.oscarcon.legacyList.updateSSIContact(user.name,
presence=ptype, show='away', status=status, ipaddr=user.icqIPaddy,
lanipaddr=user.icqLANIPaddy, lanipport=user.icqLANIPport,
icqprotocol=user.icqProtocolVersion, url=url)
                except:
                        pass

Posted by guest at 2007-08-04 20:34:17

Hi, I had the same issue and the posted solution from "guest at 2007-07-26
13:45:41" seems to work fine for me. 


Jan 12, 2008
Project Member #1 volk...@gmail.com
This may be fixed by the latest "don't crash on weird encoding" patches.
Status: Fixed