Opened 10 years ago

Closed 7 years ago

#2845 closed enhancement (fixed)

Keep Gajim's program state (data) in a separate DB

Reported by: junglecow Owned by: jim++
Priority: normal Milestone: 0.14
Component: None Version: hg
Severity: normal Keywords:
Cc: lukas@… Blocked By:
Blocking: OS: All

Description

Recently, Gajim has started storing other information than its logs inside the log DB. (Transport types, unread messages.) I think this is wrong, since we should be able to safely rename/remove logs.db without losing information. Therefore I suggest to create a new DB (gajim.db?) and keep the information there. With it, we can also add other nice things like showing full roster on startup.

Change History (22)

comment:1 Changed 10 years ago by nk

I don't understand how you cannot now remove safely logs.db ?

of course if you remove, you lose stuff in it (like unread msgs)

WONTFIX/INVALID/WORKSFORME..

comment:2 Changed 10 years ago by junglecow

The thing is that I want to store some security things which the user isn't meant to change. Of course it could still be stored in config, it's not that big…

But also, it's simply not clean programming to store things which aren't logs in logs.db. And the user does not expect side-effects from removing logs.

comment:3 Changed 10 years ago by patrys

I'd like the option to cache the roster offline. Currently if I have no connection I need to use the history browser to find the critical data.

comment:4 Changed 10 years ago by nk

in an everyday more online world, I can't find any reason to implement offline stuff in a network protocol client. Moreover you have History Manager for such stuff. imo WONTFIX

comment:5 Changed 9 years ago by steve-e

  • OS set to All

We need this for #2149. I am thinking of splitting logs.db into three databases:

  • logs.db: logs
  • cache.db: caps_cache, transports_cache and maybe roster_cache
  • gajim.db: jids, unread_messages and anything else what might be usefull one day...

comment:6 follow-up: Changed 9 years ago by asterix

jids table MUST stay with logs.db. Logs are not readable without this jids table. unread_messages can go in cache.db so we only have 2 db. One can be deleted without any pb (cache.db)

comment:7 in reply to: ↑ 6 ; follow-up: Changed 9 years ago by lukas@…

  • Cc lukas@… added

Replying to asterix:

jids table MUST stay with logs.db. Logs are not readable without this jids table. unread_messages can go in cache.db so we only have 2 db.

Perhaps we could use jid as log table's primary key instead of jid_id, which refers to the jids table. jid is unique enough (and never changes AFAIK), so it could serve as a primary key and allow us to separate jids table from logs.db.

comment:8 Changed 9 years ago by asterix

we use a separate table to store jids to use less space. For each row we use jid_id (int) instead of the long text jid.

comment:9 follow-up: Changed 9 years ago by steve-e

What do you mean with space? Total space of logs.db in byte?

Wouldn't it be faster to use jids as primary keys? No translation between jid_id and jid would be needed.

comment:10 in reply to: ↑ 9 ; follow-up: Changed 9 years ago by lukas@…

Replying to steve-e:

Wouldn't it be faster to use jids as primary keys? No translation between jid_id and jid would be needed.

That's exactly what I was saying - I think the size of logs.db is not that important now, in the age of terabyte harddrives and gigabytes of RAM.

comment:11 Changed 9 years ago by asterix

the translation is not done by reading DB each time. At startup we read jids table and keep it in mem so translation is very quick (a dict). It's why I think it's better to not duplicate the JID on every log row.

comment:12 Changed 9 years ago by steve-e

  • Milestone changed from 0.12 to 0.13

comment:14 in reply to: ↑ 7 Changed 7 years ago by Frank

Replying to lukas@…:

Perhaps we could use jid as log table's primary key instead of jid_id, which refers to the jids table. jid is unique enough (and never changes AFAIK), so it could serve as a primary key and allow us to separate jids table from logs.db.

The jid can change - e.g. changing your icq transport results in a lot of contacts with a new jid - for now you only need to change table jids to compansate this. If you put jid as key you need to change the hole logs table. Or think of a contact switching his jid and abandoning his old jid and now I don't want to carry two jids of him with me. Now I need to change only jid in jids table and everything is ok. Another reason not to have jid key: searching logs with int as identifier is faster than searching with text as identifier, beside the space requirements on disc and in ram.

comment:15 in reply to: ↑ 10 Changed 7 years ago by Frank

Replying to lukas@…:

Replying to steve-e:

Wouldn't it be faster to use jids as primary keys? No translation between jid_id and jid would be needed.

That's exactly what I was saying - I think the size of logs.db is not that important now, in the age of terabyte harddrives and gigabytes of RAM.

But think of handhelds, smart phones, netbooks and such devices. They usually don't have terrabyte of disc space or gigabytes of ram. My logs.db would be about 1/3 (about 10MB) bigger if changing from jid_id to jids bases storing. It isn't not only about disc space or available ram. The log needs to be parsed, text parsing consumes more cpu cycles than simple fetching integers like jid_id. More cpu cycles means longer waiting for results, energy consumptions rises, shorter battery life time and so on. And you would store a lot of redundand data for nothing - you won't win a dime if changing jid_id to jids.

comment:16 Changed 7 years ago by Jim++

Replying to asterix:

jids table MUST stay with logs.db. Logs are not readable without this jids table. unread_messages can go in cache.db so we only have 2 db. One can be deleted without any pb (cache.db)

There is actually a problem with that. roster_entry table, roster_group table, unread_messages table (and maybe transports_cache table ?) use jid_id, so they can't be really safely separated from jid table.

comment:17 Changed 7 years ago by Jim++

13:39:50 Steve-e: I think unread messages should stay with logs.db
13:39:57 Steve-e: for the rest
13:40:06 Steve-e: I think it is not a big problem to move it to cache.db
13:40:12 Steve-e: just replace the jid_id with the real jid
13:40:15 Steve-e: yes this is longer
13:40:21 Steve-e: but we only have one entry for each jid
13:40:28 Steve-e: so there is no real overhead here

Seems ok to me.

comment:18 Changed 7 years ago by asterix

Shouldn't unread_messages table stay in logs.db? It's not really nice to loose this table if ~/.cache is removed

comment:19 Changed 7 years ago by johnny

  • Milestone changed from 0.13 to 0.14

i don't think we're goingto do this for 0.13 are we? seems too intrusive for now

comment:20 Changed 7 years ago by asterix

  • Owner changed from asterix to jim++

comment:21 Changed 7 years ago by asterix

  • Blocking 2149 added

comment:21 Changed 7 years ago by asterix

  • Resolution set to fixed
  • Status changed from new to closed

(In [3372b574e372]) split logs.db into logs.db and cache.db. Fixes #2845

comment:22 Changed 7 years ago by asterix

  • Blocking 2149 removed
Note: See TracTickets for help on using tickets.