Bug #1307
closeddatabase locked during update
Added by OleS over 11 years ago. Updated over 11 years ago.
100%
Description
Hi there,
I'm still experiencing database locks while updateing epg.db.
It occures to me this is due to the amount of data being requested. Right now I'm down to 2 days of epg in advance and 38 channels resulting in ~3500 xml events and ~200 vdr events.
Maybe you could shed some light on how much data plugin and/or database is able to handle.
TIA and cheers,
Ole
Updated by OleS over 11 years ago
Hi again,
seems like I'm still experiencing database locks. Now even with ~3500 xml events...
Help would be greatly appreciated.
Cheers,
Ole
Updated by OleS over 11 years ago
PS: I'm running 0.2.0pre.git20121006-0yavdr2~precise as delivered by yaVDR 0.5 atm.
Cheers,
Ole
Updated by OleS over 11 years ago
Me again :)
according to this admittedly very old link --> http://weblogs.asp.net/jgalloway/archive/2005/07/08/418472.aspx
it might be due to the locking algorithms used by sqlite. Is xmltv2vdr able to use another backend?
Cheers,
Ole
Updated by Joe_D over 11 years ago
- Tracker changed from Support to Bug
- Status changed from New to Assigned
- Assignee set to Joe_D
please try latest git!
Updated by OleS over 11 years ago
Hi, output of latest GIT version:
root@htpc:~/install/xmltv2vdr/vdr-plugin-xmltv2vdr# tail -f /var/log/syslog | grep -i xmltv2vdr
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr importer thread started (pid=8959, tid=9576)
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' epgdata2xmltv 14 'X' 0 13th-street.de 3sat.de animax.de anixe.de ard.de arte.de axntv.de das-vierte.de einsextra.de einsfestival.de einsmuxx.de fox-channel.de history-hd.de kabel1.de mgm.de orf1.at orf2.at prosieben.de rtl.de rtl2.de sat1.de sci-fi.de sixx.de sky-action.de sky-atlantic-hd.de sky-cinema-1.de sky-cinema-24.de sky-cinema-hits.de sky-cinema.de superrtl.de tele5.de tnt-film.de tnt-serie.de vox.de zdf.de zdfinfo.de zdfkultur.de zdfneo.de
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' executing epgsource
Apr 19 10:41:04 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' parsing output
Apr 19 10:41:05 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:05 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:05 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:06 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:09 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' processed 16838 xmltv events
Apr 19 10:41:10 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' importing from db
Apr 19 10:41:12 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' processed 7259 vdr events
Apr 19 10:41:12 htpc vdr: [9576] xmltv2vdr importer thread ended (pid=8959, tid=9576)
Cheers,
Ole
Updated by OleS over 11 years ago
Hi again,
please ignore my previous post, compiled the wrong version...
Unfortunately, the new git version won't compile:
root@htpc:~/install/vdr-plugin-xmltv2vdr# make VDRDIR=/usr/include/vdr LIBDIR=/usr/lib/vdr/plugins
g++ -g -Wall -Woverloaded-virtual -Wno-parentheses -O2 -fPIC -c -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_XOPEN_SOURCE -DPLUGIN_NAME_I18N='"xmltv2vdr"' -I/usr/include/dvb-s2api-liplianin -I/usr/include/vdr/include -I/usr/include/libxml2 xmltv2vdr.cpp
xmltv2vdr.cpp: In Elementfunktion »bool cEPGHandler::check4proc(cEvent*, cTimer**, cEPGMapping**)«:
xmltv2vdr.cpp:431:49: Fehler: keine passende Funktion für Aufruf von »cTimers::GetMatch(cEvent*&, eTimerMatch*)«
xmltv2vdr.cpp:431:49: Anmerkung: Kandidaten sind:
/usr/include/vdr/include/vdr/timers.h:121:11: Anmerkung: cTimer* cTimers::GetMatch(time_t)
/usr/include/vdr/include/vdr/timers.h:121:11: Anmerkung: Kandidat erwartet 1 Argument, 2 angegeben
/usr/include/vdr/include/vdr/timers.h:122:11: Anmerkung: cTimer* cTimers::GetMatch(const cEvent*, int*)
/usr/include/vdr/include/vdr/timers.h:122:11: Anmerkung: keine bekannte Umwandlung für Argument 2 von »eTimerMatch*« nach »int*«
make: *** [xmltv2vdr.o] Fehler 1
Cheers,
Ole
Updated by Joe_D over 11 years ago
- % Done changed from 0 to 10
ok you are using an "outdated" vdr version ;)
please change
eTimerMatch TimerMatch = tmNone; *timer=Timers.GetMatch(event,&TimerMatch)
into
#if VDRVERSNUM<=10732 int TimerMatch = tmNone; #else eTimerMatch TimerMatch = tmNone; #endif *timer=Timers.GetMatch(event,&TimerMatch)
in xmltv2vdr.cpp line 429ff
Updated by OleS over 11 years ago
Hi,
on first glance, the new version seems ok:
Apr 19 13:45:29 htpc vdr: [25511] initializing plugin: xmltv2vdr (0.2.0pre): Importiert xmltv epg in den VDR
Apr 19 13:45:29 htpc vdr: [25511] starting plugin: xmltv2vdr
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using codeset 'UTF-8'
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using file '/srv/vdr/video.00/epg.db' for epg database
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using dir '/var/cache/vdr/epgimages' for epgimages (7)
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' added epgsource
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' reading source config
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is providing data through a pipe
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' updates data @00:00
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is needing a pin
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is providing pics
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' daysmax=14
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' reading plugin config
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' pin set
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' daysinadvance=3
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' using pics=0
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' weekdays=MTWTFSS
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' nextrun on Sat Apr 20 03:15:00 2013
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is ready2parse
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using sqlite v3.7.9
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr importer thread started (pid=25511, tid=25644)
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' epgdata2xmltv 3 'X' 0 13th-street.de 3sat.de animax.de anixe.de ard.de arte.de axntv.de das-vierte.de einsextra.de einsfestival.de einsmuxx.de fox-channel.de history-hd.de kabel1.de mgm.de orf1.at orf2.at prosieben.de rtl.de rtl2.de sat1.de sci-fi.de sixx.de sky-action.de sky-atlantic-hd.de sky-cinema-1.de sky-cinema-24.de sky-cinema-hits.de sky-cinema.de superrtl.de tele5.de tnt-film.de tnt-serie.de vox.de zdf.de zdfinfo.de zdfkultur.de zdfneo.de
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' executing epgsource
Apr 19 13:45:48 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' parsing output
Apr 19 13:45:51 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' processed 4181 xmltv events
Apr 19 13:45:51 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' importing from db
but 'importing from db' seems not. Importing this amount of data usually took
no more than 2-3 minutes. Now I'm up to 15 and still counting. Is this an expected
behaviour on your side?
Just to understand the process: importing means merging /srv/vdr/video.00/epg.db
into /var/cache/vdr/epg.data, or am I mistaken?
Cheers,
Ole
Updated by OleS over 11 years ago
Update: after 20 minutes the import finished with no error.
Conclusion: it's slow but it works!
Thanks and cheers,
Ole
Updated by Joe_D over 11 years ago
- % Done changed from 10 to 90
I missed to remove a usleep in the code, please try the latest GIT!
Updated by Joe_D over 11 years ago
- Status changed from Assigned to Closed
- % Done changed from 90 to 100