Log migration throught dialog window doesn't work

[multani]

When running an old Gajim installation, which doesn't have a SQLite database, Gajim display a window telling me to wait while logs are migrated. But after 15min., nothing happens : no output in textview, no output on stdout, and I have only 650ko of logs.

Running the migration script manually work correctly.

[multani]

A little investigation show that the t thread at the end of src/gajim.py has some problem :

Index: gajim.py
===================================================================
--- gajim.py    (révision 4755)
+++ gajim.py    (copie de travail)
@@ -1523,9 +1523,12 @@
                        import Queue
                        q = Queue.Queue(100)
                        m = migrate_logs_to_dot9_db.Migration()
+                       print "Migrating logs"
                        dialog = dialogs.ProgressDialog(_('Migrating Logs...'), _('Please wait while logs are being migrated...'), q)
                        t = threading.Thread(target = m.migrate, args = (q,))
+                       print "Starting thread"
                        t.start()
+                       print "Thread started"
                        gobject.timeout_add(500, wait_migration, m)
                        gtk.main()
                        # Init logger values (self.con/cur, jid_already_in)
Index: common/migrate_logs_to_dot9_db.py
===================================================================
--- common/migrate_logs_to_dot9_db.py   (révision 4755)
+++ common/migrate_logs_to_dot9_db.py   (copie de travail)
@@ -169,6 +169,7 @@
                        self.con.commit()

        def migrate(self, queue = None):
+               print "In Migration::migrate"
                self.queue = queue
                self.con = sqlite.connect(PATH_TO_DB)
                os.chmod(PATH_TO_DB, 0600) # rw only for us

I give this ouput (after several minutes) :

$ python gajim.py
Migrating logs
Starting thread
Thread started

I got only once the "In Migration::migrate" text on stdout, but I can't reproduce it. Maybe a Python bug :/ So, here is my config (Debian SID x86) : python 2.3.5-3 python-pysqlite2 2.0.5-1 python-gtk2 2.6.3-2

If you need more, you can tell me.

[multani]

Works as expected on Windows ...

[asterix]

it works well here.

It's normal that migrate is called only once. Could you try with more print? For exemple to see if visit is called ? if PATH_TO_LOGS_BASE_DIR is correct ?

Could you come in room so we can talk ?

[gjc@…]

I think gdb's help would be welcome here. If you could attach gdb to your process (gdb python <pid-of-gajim>), then:

(gdb) thread apply all bt

And post the stack trace..

[multani]

Here is the backtrace :

(gdb) thread apply all bt

Thread 2 (Thread -1271366736 (LWP 5940)):
#0  0xb7f699db in sem_wait@GLIBC_2.0 () from /lib/tls/libpthread.so.0
#1  0xb4387b90 in ?? ()
#2  0x080dd04e in PyThread_acquire_lock ()
#3  0x080b19e1 in PyEval_RestoreThread ()
#4  0x080d7599 in PyGILState_Ensure ()
#5  0x080e0b1d in _PyObject_GC_NewVar ()
#6  0xb7f65cfd in start_thread () from /lib/tls/libpthread.so.0
#7  0xb7ecd19e in clone () from /lib/tls/libc.so.6

Thread 1 (Thread -1210079552 (LWP 5938)):
#0  0xb7ec2da9 in poll () from /lib/tls/libc.so.6
#1  0xb790757b in g_main_context_check () from /usr/lib/libglib-2.0.so.0
#2  0xb7907bd8 in g_main_loop_run () from /usr/lib/libglib-2.0.so.0
#3  0xb75d1629 in gtk_main () from /usr/lib/libgtk-x11-2.0.so.0
#4  0xb787d46a in init_gtk () from /usr/lib/python2.3/site-packages/gtk-2.0/gtk/_gtk.so
#5  0x080b6c0d in PyEval_GetFuncName ()
#6  0x080b8357 in PyEval_EvalCodeEx ()
#7  0x080b85d5 in PyEval_EvalCode ()
#8  0x080d929c in PyRun_FileExFlags ()
#9  0x080d9563 in PyRun_SimpleFileExFlags ()
#10 0x08054f07 in Py_Main ()
#11 0xb7e10ed0 in __libc_start_main () from /lib/tls/libc.so.6
#12 0x080548e1 in _start ()
#0  0xb7ec2da9 in poll () from /lib/tls/libc.so.6
(gdb)

[Gustavo Carneiro <gjc@…>]

Looks like the wrapper for gtk.main() didn't release the Python GIL properly. As to why this happens, I can't say at this point...

[Gustavo Carneiro <gjc@…>]

In src/gajim.py:

import gobject
if sys.version[:4] >= '2.4': # FIXME: remove me when we abandon python23
        gobject.threads_init()

So, basically, you avoid calling gobject.threads_init() in python 2.3 in order to prevent a weird crash. But if you don't call gobject.threads_init(), pygtk doesn't know this is a threaded app and so doesn't bother with releasing the GIL.

[nk]

that weird py23/pygtk bug is very strange and it's not us. since it's crash though, I think it's better we take our chances and leave that check in, as unfortunately debian unstable stills ships py23 (the rest of the big distros ship py24)

one other solution is to find and fix the bug in pygtk + python23 but even if that is fixed ppl have to update to that pygtk etc

we can't do much. I choose to take the risk, rather than almost always crash . I propose CANTFIX/INVALID/WORKSFORME

[asterix]

what can be easily done to not indefinitively wait is to add a check: if 10 seconds after migration started, we get nothing from migration through the queue, we show a dialog: "migration failed, run script manually" and then close gajim.

[asterix]

done

[multani]

I tested it, and it works merely as supposed .... ... but when clicking on the Ok button of the "Migration failed" dialog box, the migration process starts ...

When commenting the ", args = (q,)" in threading.Thread, I can see the output of the log processing, which should be displayed normally in the first window ...