Sound support improvements for Gajim

[ressu@…]

I've remodeled the sound support in Gajim to use an object based system. This allows easier implementing of native sound support. In my case i added GStreamer support in addition to the external players. This should reduce the latency when playing sound events on laptops when the disk is in sleep mode. Also it should make things more portable.

[ressu@…]

Patch to remodel the sound support

[ressu@…]

This also fixes the problem of sound events queuing indefinitely when

sound output is reserved.

There is one problem with this patch though. I didn't create a GUI to select the backend. I don't know if it's a big issue anyway. GStreamer is quite capable of playing files and in windows the winsound library will take care of playing.

I think it might even be wise to drop the player configuration from the configuration dialogs and leave it as a hidden option for those who want to customise it (i don't think there are too many people who will)

[dkirov]

This patch is great, but I when I send messages with it there is a little

delay. Do you know what can be the reason ?

[dkirov]

Also there are is a small TB:

 Traceback (most recent call last):
   File "/home/kirov/Projects/gajim/trunk/src/common/xmpp/idlequeue.py",
 line 133 , in process_events
     obj.pollin()
   File
 "/home/kirov/Projects/gajim/trunk/src/common/xmpp/transports_nb.py", line
 144, in pollin
     self._do_receive()
   File
 "/home/kirov/Projects/gajim/trunk/src/common/xmpp/transports_nb.py", line
 242, in _do_receive
     self.on_receive(received)
   File
 "/home/kirov/Projects/gajim/trunk/src/common/xmpp/dispatcher_nb.py", line
 349, in dispatch
     handler['func'](session,stanza)
   File
 "/home/kirov/Projects/gajim/trunk/src/common/connection_handlers.py", lin
 e 1328, in _messageCB
     self.dispatch('GC_MSG', (frm, msgtxt, tim))
   File "/home/kirov/Projects/gajim/trunk/src/common/connection.py", line
 98, in dispatch
     self.put_event((event, data))
   File "/home/kirov/Projects/gajim/trunk/src/common/connection.py", line
 94, in put_event
     gajim.handlers[ev[0]](self.name, ev[1])
   File "gajim.py", line 871, in handle_event_gc_msg
     gc_control.on_message(nick, array[1], array[2])
   File "/home/kirov/Projects/gajim/trunk/src/groupchat_control.py", line
 450, in  on_message
     self.print_conversation(msg, nick, tim)
   File "/home/kirov/Projects/gajim/trunk/src/groupchat_control.py", line
 548, in  print_conversation
     sound.play_event('muc_message_highlight')
 AttributeError: 'str' object has no attribute 'play_event'

[ressu@…]

There is an initial delay on the first message, that is caused by the

GStreamer init. The actual player object (and the GStreamer libs) are created with the first event played. Other than the first event being slightly delayed, it was at least as fast as the external player.

The error you are seeing, is caused by the sound variable in that function. I'll update the patch in a moment.

[ressu@…]

Updated version of the patch, created from r6355

[dkirov]

Thanks and sorry for the delay. I'll apply it in svn ASAP. Is video sink

really needed ? Will the latency be reduced if we use only audio sink ?

[dkirov]

I also noticed that with this patch gajim forks separate proccess, which

make it far more clumsy and memory hungry. If this fork cannot be avoided, then we'll make an ACE option 'use_gstreamer', which will be False by default.

[ressu@…]

Video sink is there to prevent video from showing if someone decides to

use a video file as a notification. It's not actually used unless there actually is video data inside the stream, so basically it's a sane fallback.

There should be no forking. Gstreamer will use a separate thread to play the sound file, which allows gajim to stay responsive while the sound file is playing without doing any trickery. If you still think that this is unacceptable as default, i'll modify the patch to use an ACE option.

[ressu@…]

Hmm.. It looks like gstreamer never releases the sound device. Without

dmix, this will cause problems. I'll see if i can modify the patch so that that doesn't happen..

[dkirov]

...and the last played file too... When I set default sink to 'esd' it

doesn't fork anymore.

[ressu@…]

Ok, I'm uploading a new version of the patch in a moment. Now

GajimGstPlayer? cleans up when the stream is done playing (previously it was just left at the state it was in).

Also, are you certain that you are seeing forks and not threads? Threads share the memory with the main process so threads should be ok (as long as they are handled properly, which they are) because there is little memory overhead which is mostly just caused by the gstreamer libraries themselves.

[ressu@…]

Updated diff with gstreamer cleanups, diffed against r6414

[ressu@…]

Updated gstreamer patch (again ;) This time with ACE support.

[nk]

Dimitur, Ressu, what's wrong with forking, if the proc dies when the snd

is played?

[ressu@…]

I think he is referring to cleaning up after forks as well as IPC issues.

But since this is gstreamer library handling all this work to handle the separate threads, there is no memory thread safety issues or IPC issues. And since python does cleanup on exit, it will call the destructors required to clean up after exit.

I don't see a problem.. ;)

Other than that, i have an updated patch for the latest CVS that handles the recently added play_soundfile() function too (deletes it too), i'll upload it if there is need. there are no other changes in the patch.

[dkirov]

nk:

One problem is that some systems maynot have the command "aplay". I tried to use "play" instead, but it didn't worked with multichanels (while rhythmbox, or quotlibet is playing, gajim is unable to play sounds). I think using gstreamer is good, but it is true that "reducing latency" is yet not achiavable with it.

[patrys@…]

Could the gstreamer done event be modified to use an internal sound queue and instead of shutting down the sound subsystem, play another file (then the play command itself would have to check the gstbin state and queue instead of play if it's in use).

[ressu@…]

It should be easy to create such a queue. There is an event triggered when a file stops playing, just by adding a check if there is a file queued to that event would do the trick.

The reason why this wasn't implemented in my patch is that i don't think sound events should be queued. There is no point in playing sequential events since events are supposed to draw your attention to something. Sequential events loose their purpose after the first event.

Also, one of the reasons why i wrote the original patch was to get rid of aplay playing multiple events when the sound card was busy.

The current patch has quite a bit of conflicts when applied against the current SVN. I haven't been updating it anymore.

[asterix]

indeed a queue sounds bad, is there someone who can't play 2 sounds at the same time ? Everyone has esd or somethings like that, am I wrong ?

[ressu@…]

Gstreamer is capable of playing multiple sounds at the same time either with hardware or software mixing. I decided to go with single event at once model due to 2 reasons. One is that it makes the implementation quite a bit more complex. When tearing down the pipeline for the parts that have finished playing (so we don't leak memory) we need to keep track of which sound event finished playing, which parts of the pipeline need to be taken down and conditionally rebuild the pipeline.

The other reason is that i don't think that most of the event sounds mix well with each other.

That said, it shouldn't be too hard to adapt the patch to multiple events at once model since gstreamer already provides the required information with the internal events. Only thing that needs to be written is the logic.

[asterix]

I understand your point, but imagine you get in the same time a contact that signs in, and 0.5 second later a message, in this case you won't hear the message sound, and if you only listen to sounds and don't look at (or don't have) systray, you may miss it ...

[patrys]

Actually you will hear just the message sound as with each event the gst playbin gets stopped and starts playing another sound.

[ressu@…]

The only case where it could be that a message notification is missed is when you receive a message immediately followed by a signon or signoff event. The message notification will be cut short.

[patrys]

This shouldn't be a problem as both esdplay (which I use) and aplay (with dmix enabled) would result in a similar scenario.

[patrys]

Can anyone support me on this one? It would certainly help.

http://bugzilla.gnome.org/show_bug.cgi?id=374073