UnicodeDecodeError

[quinnman]

I am using the Windows version 0.11. Previously there was no problem, but after installing the new version I am unable to run it with UnicodeDecodeError? error. I tried to remove the configuration file, but with no luck (by the way, were is the configuration file stored in Win). Direct uninstalling of 0.11 version and installing 0.10.1 version works, so I am stuck with the older version. I also tried the version with/without GTK.

[Jim++]

Could you give the exact error message please ? In gajim.exe.log.

Maybe this one ? :

15:41:16 cédric: Traceback (most recent call last):
  File "gajim.py", line 29, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 64, in <module>
  File "common\logger.pyc", line 32, in <module>
  File "common\configpaths.pyc", line 104, in <module>
  File "common\configpaths.pyc", line 90, in init
  File "ntpath.pyc", line 368, in expanduser
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 27-29: invalid dat

[anonymous]

Replying to Jim++:

Could you give the exact error message please ? In gajim.exe.log. Maybe this one ? : {{{ 15:41:16 cédric: Traceback (most recent call last): File "gajim.py", line 29, in <module> File "message_control.pyc", line 14, in <module> File "gtkgui_helpers.pyc", line 28, in <module> File "vcard.pyc", line 27, in <module> File "common\helpers.pyc", line 29, in <module> File "common\gajim.pyc", line 64, in <module> File "common\logger.pyc", line 32, in <module> File "common\configpaths.pyc", line 104, in <module> File "common\configpaths.pyc", line 90, in init File "ntpath.pyc", line 368, in expanduser UnicodeDecodeError?: 'utf8' codec can't decode bytes in position 27-29: invalid dat }}}

That is almost identical the error message I encountered

Traceback (most recent call last):
  File "gajim.py", line 29, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 64, in <module>
  File "common\logger.pyc", line 32, in <module>
  File "common\configpaths.pyc", line 104, in <module>
  File "common\configpaths.pyc", line 90, in init
  File "ntpath.pyc", line 368, in expanduser
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 32-36: unexpected end of data

[asterix]

I'll build a new installer with os.path.expanduser('~') instead of os.path.expanduser(u'~') this evening. We'll see if that fix the problem

[anonymous]

Replying to asterix:

I'll build a new installer with os.path.expanduser('~') instead of os.path.expanduser(u'~') this evening. We'll see if that fix the problem

Thanks, I am looking forward to trying it!

[anonymous]

Still no luck. Not sure if tried the version you mentioned (ver. 0.11-2), but I still receive the same error:

Traceback (most recent call last):
  File "gajim.py", line 29, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 64, in <module>
  File "common\logger.pyc", line 32, in <module>
  File "common\configpaths.pyc", line 104, in <module>
  File "common\configpaths.pyc", line 90, in init
  File "ntpath.pyc", line 368, in expanduser
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 32-36: unexpected end of data

[nk]

hi

I'm not sure if you have python installed, but if you can please do so.

then please launch python and there type

import os.path

os.path.expanduser('~')

thank you

(Yann no need to u'~' as the problem seems to be in Python STDLIB! [I hope you produced the installer with latest stable of the py25 series)

[anonymous]

Replying to nk:

hi I'm not sure if you have python installed, but if you can please do so. then please launch python and there type import os.path os.path.expanduser('~') thank you (Yann no need to u'~' as the problem seems to be in Python STDLIB! [I hope you produced the installer with latest stable of the py25 series)

Hallo again. Here is what I received:

'C:\\Document and Settings\\Jan Fr\xfdbort'

IMHO this looks like the source of the problem. Somehow Python does not read my name correctly as it contains same strange character 'ý'.

[asterix]

I've not built it yet, this evening when I'll be at home :)

[junglecow]

Replying to nk:

(Yann no need to u'~' as the problem seems to be in Python STDLIB! [I hope you produced the installer with latest stable of the py25 series)

Yes, this does indeed seem to be a problem of stdlib and/or Windows. The problem is that os.expanduser seems to expect utf-8, but the environment variable that it reads is set in a different codepage.

I'm sure it's possible to configure the default encoding of Python somewhere. However, it's possible that such settings are also compiled into Yann's binary, so it's his default codepage, not of the user installing the package.

Anyway, seems we can't rely on os.expanduser() to do the right thing in Windows. I'll patch and try filesystemencoding instead.

[junglecow]

(In [7704]) [Trunk]

• In verbose mode, print encodings. (Especially for Windows users who don't have Python) (gajim.py)
• Attempt at fixing traceback when getting user's home directory in Windows. See #2812. (c/configpaths.py)
• Show 'error' icon next to account while waiting for reconnect. Fixes #2786. (c/connection_handlers.py, c/gajim.py, c/connection.py)

[PyOpenSSL]

• Fix 100% CPU usage and hanging connection when server closes connection on us. (c/x/transports_nb.py)
• Fix 'hanging' connection when server closes the connection on us before we can open the XML stream. (Disconnect handler didn't get called.) (c/x/client_nb.py)
• Change prints to logger calls, various enhancements to debug printing, reduce spam (c/x/transports_nb.py)
• this ?\226?\134?\146 self (c/x/transports_nb.py)
• Call _do_receive() once to collect error message from socket, when error flag is raised in scheduler. (c/x/transports_nb.py)

[junglecow]

• See: http://bugzilla.gnome.org/show_bug.cgi?id=132040
• And: https://launchpad.net/distros/ubuntu/+source/pygtk/+bug/27112

Thank you, PyGTK :-(

[asterix]

could you test this one ?

[Luke_Mica]

I have this problem as well - I installed the package from the message above. The program starts and works smoothly until I decide to close dialogue windows - then it displays the error message and refers to gajim.exe.log.

This is what I found in the log:

[Errno 2] No such file or directory: u'C:\\Documents and Settings\\\u0139\x81uKa$z3K\\Dane aplikacji\\Gajim\\.Config'

(ofc. Dane aplikacji = Application data)

[asterix]

could you try this other one ?

[quinnman]

Replying to asterix:

could you try this other one ?

This time I could import an existing account, but when gajim tried to log in, it crashed wit this message in gajim.exe.log

[Errno 2] No such file or directory: u'C:\\Documents and Settings\\Jan Fr\u0102\u02ddbort\\Data aplikac\u0102\xad\\Gajim\\.Config'

[junglecow]

(In [7710]) - Use bytestrings instead of unicode for filesystem paths. See #2812 and close if fix works.

[asterix]

and this one ?

[Luke Mica]

no success...

that's what it saves in the log while startup...

NameError: global name 'fse' is not defined

[junglecow]

Ouch. Asterix, please take verbatim version of configpaths from trunk, it should work.

[junglecow]

Marked #2828 as dup of this bug

[asterix]

one more try

[anonymous]

The requested URL /gajim-0.11+gtk-test4.exe was not found on this server.

[asterix]

now it is, it was a G instead of a g :/

[Luke Mica]

another line in the log...

sqlite3.OperationalError: unable to open database file

[nk]

Gustavo, can u help us with this mess?

[asterix]

Luke, could you give us the full Traceback please ?

[asterix]

I did a new one from home. Please give us the full Traceback

[Luke Mica]

Traceback (most recent call last):
  File "gajim.py", line 39, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 65, in <module>
  File "common\logger.pyc", line 89, in __init__
  File "common\logger.pyc", line 117, in init_vars
  File "common\logger.pyc", line 103, in open_db
sqlite3.OperationalError: unable to open database file

[asterix]

one more :) This time we use the same code as in 0.10.1

[anonymous]

and now the same problem as with the first test package: while closing the dialogue window it crashes and saves in the log:

[Errno 2] No such file or directory: u'C:\\Documents and Settings\\\u0139\x81uKa$z3K\\Dane aplikacji\\Gajim\\.Config'

[asterix]

full Traceback please

[asterix]

this time we use unicode only for sqlite.

[asterix]

sorry it's http://www.gajim.org/gajim-0.11+gtk-test7.exe

[anonymous]

:( got a error :( after instalation:

Traceback (most recent call last):
  File "gajim.py", line 39, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 65, in <module>
  File "common\logger.pyc", line 90, in __init__
  File "common\logger.pyc", line 118, in init_vars
  File "common\logger.pyc", line 104, in open_db
sqlite3.OperationalError: unable to open database file

[asterix]

this one doesn't change anything, it just adds some print in gajim.exe.log. could you give us the result please. (remove gajim.exe.log before launching gajim to have a fresh one)

[Andry]

configpaths.gajimpaths['LOG_DB'] C:\Documents and Settings\Андрей\Application Data\Gajim\Logs.db
sys.getfilesystemencoding() mbcs
LOG_DB_PATH C:\Documents and Settings\Андрей\Application Data\Gajim\Logs.db
join_all C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
join_all2 C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
join_all3 C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
join in 2 steps C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
join in 2 steps2 C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
join in 2 steps3 C:\Documents and Settings\Андрей\Application Data\Gajim\logs.db
Traceback (most recent call last):
  File "gajim.py", line 40, in <module>
  File "message_control.pyc", line 14, in <module>
  File "gtkgui_helpers.pyc", line 28, in <module>
  File "vcard.pyc", line 27, in <module>
  File "common\helpers.pyc", line 29, in <module>
  File "common\gajim.pyc", line 65, in <module>
  File "common\logger.pyc", line 100, in __init__
  File "common\logger.pyc", line 128, in init_vars
  File "common\logger.pyc", line 114, in open_db
sqlite3.OperationalError: unable to open database file

new error_log :( i`m Use in WinXP RUS UserName?.

[junglecow]

Hmmm, after studying source code, I think that connect wants utf-8 strings. (Thank you, pysqlite documentation, for mentioning this :-( ) See pysqlite_connection_init, and in particular line 81 where sqlite3_open is called. Then, see sqlite.h (scroll down to sqlite3_open) and read comments there.

Not sure if that's all of it (I doubt that environment variables are mbcs), but I hope it helps some.

If all else fails, we can use the workaround here.

[anonymous]

If all else fails, we can use the workaround here.

Waiting for it.

[asterix]

here is a version with the workaround.

[Dicson]

Replying to asterix:

here is a version with the workaround.

Don't work. i`m Use in WinXP RUS

[asterix]

TB is still the same ? that still fail in logger.py in open_db() ? which line exactly ?

please give the full traceback

[Dicson]

Traceback (most recent call last):

File "gajim.py", line 1904, in process_connections File "common\xmpp\idlequeue.pyc", line 209, in process File "common\xmpp\transports_nb.pyc", line 153, in pollin File "common\xmpp\transports_nb.pyc", line 255, in _do_receive File "common\xmpp\dispatcher_nb.pyc", line 353, in dispatch File "common\connection_handlers.pyc", line 1094, in _vCardCB File "gajim.py", line 1256, in save_avatar_files

gobject.GError: ¦ЯTА¦-¦¬¦¬¦-TИTС¦¬ TБ¦-¦-¦¦ ¦¬TА¦¬ ¦-TВ¦¦TАTЛTВ¦¬¦¬ TД¦-¦¦¦¬¦- "C:\Documents and Settings\я¬-я¬-я¬-я¬-я¬-\Application Data\Gajim\Avatars\fominde@…" ¦+¦¬TП ¦¬¦-¦¬¦¬TБ¦¬: Invalid argument Traceback (most recent call last):

File "roster_window.pyc", line 1250, in show_tooltip File "tooltips.pyc", line 127, in show_tooltip File "tooltips.pyc", line 400, in populate

ValueError?: image should be a GdkPixbuf? or empty Traceback (most recent call last):

File "roster_window.pyc", line 1250, in show_tooltip File "tooltips.pyc", line 127, in show_tooltip File "tooltips.pyc", line 400, in populate

ValueError?: image should be a GdkPixbuf? or empty Traceback (most recent call last):

File "roster_window.pyc", line 1250, in show_tooltip File "tooltips.pyc", line 127, in show_tooltip File "tooltips.pyc", line 400, in populate

ValueError?: image should be a GdkPixbuf? or empty

[anonymous]

Cant close gajim ( I think it cant save the settings in folder with Cirilic name :(

Traceback (most recent call last):
  File "systray.pyc", line 268, in on_quit_menuitem_activate
  File "roster_window.pyc", line 3346, in on_quit_menuitem_activate
  File "roster_window.pyc", line 3278, in quit_gtkgui_interface
  File "message_window.pyc", line 794, in shutdown
  File "gajim.py", line 1908, in save_config
  File "common\optparser.pyc", line 123, in write
WindowsError: [Error 2]

[asterix]

who is anonymous ? what version did you test ?

[Andry]

i tested http://www.gajim.org/gajim-0.11+gtk-test9.exe this verison. ICQ: 5223069

[asterix]

ok, after a looooooong try with Dicson, here is a version that should work.

do you confirm it works ?

[quinnman]

Replying to asterix:

ok, after a looooooong try with Dicson, here is a version that should work. do you confirm it works ?

Finally it works! Thank you for your effort :-)

[asterix]

(In [7759]) handle correctly non ascii chars in path under win. fixes #2812

[asterix]

(In [7760]) new windows binary that fix unicode problem. see #2812