Project

General

Profile

Actions
Copy link

Bug #1307

closed

database locked during update

Added by OleS over 11 years ago. Updated over 11 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Joe_D
Target version:
-
Start date:
03/22/2013
Due date:
% Done:

100%

Estimated time:

Description

Hi there,

as mentioned here:
http://www.vdr-portal.de/board16-video-disk-recorder/board99-distributionen/board96-yavdr/p1132110-0-5-xmltv2vdr-bringt-database-is-locked/?highlight=database+locked#post1132110

I'm still experiencing database locks while updateing epg.db.

It occures to me this is due to the amount of data being requested. Right now I'm down to 2 days of epg in advance and 38 channels resulting in ~3500 xml events and ~200 vdr events.
Maybe you could shed some light on how much data plugin and/or database is able to handle.

TIA and cheers,
Ole

Actions
Copy link
 #1

Updated by OleS over 11 years ago

Hi again,
seems like I'm still experiencing database locks. Now even with ~3500 xml events...
Help would be greatly appreciated.

Cheers,
Ole

Actions
Copy link
 #2

Updated by OleS over 11 years ago

PS: I'm running 0.2.0pre.git20121006-0yavdr2~precise as delivered by yaVDR 0.5 atm.

Cheers,
Ole

Actions
Copy link
 #3

Updated by OleS over 11 years ago

Me again :)

according to this admittedly very old link --> http://weblogs.asp.net/jgalloway/archive/2005/07/08/418472.aspx

it might be due to the locking algorithms used by sqlite. Is xmltv2vdr able to use another backend?

Cheers,
Ole

Actions
Copy link
 #4

Updated by Joe_D over 11 years ago

  • Tracker changed from Support to Bug
  • Status changed from New to Assigned
  • Assignee set to Joe_D

please try latest git!

Actions
Copy link
 #5

Updated by OleS over 11 years ago

Hi, output of latest GIT version:

root@htpc:~/install/xmltv2vdr/vdr-plugin-xmltv2vdr# tail -f /var/log/syslog | grep -i xmltv2vdr
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr importer thread started (pid=8959, tid=9576)
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' epgdata2xmltv 14 'X' 0 13th-street.de 3sat.de animax.de anixe.de ard.de arte.de axntv.de das-vierte.de einsextra.de einsfestival.de einsmuxx.de fox-channel.de history-hd.de kabel1.de mgm.de orf1.at orf2.at prosieben.de rtl.de rtl2.de sat1.de sci-fi.de sixx.de sky-action.de sky-atlantic-hd.de sky-cinema-1.de sky-cinema-24.de sky-cinema-hits.de sky-cinema.de superrtl.de tele5.de tnt-film.de tnt-serie.de vox.de zdf.de zdfinfo.de zdfkultur.de zdfneo.de
Apr 19 10:38:00 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' executing epgsource
Apr 19 10:41:04 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' parsing output
Apr 19 10:41:05 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:05 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:05 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:06 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:07 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9042] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:08 htpc vdr: [9039] xmltv2vdr: ERROR sqlite3: 5 database is locked (par)
Apr 19 10:41:09 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' processed 16838 xmltv events
Apr 19 10:41:10 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' importing from db
Apr 19 10:41:12 htpc vdr: [9576] xmltv2vdr: 'epgdata2xmltv' processed 7259 vdr events
Apr 19 10:41:12 htpc vdr: [9576] xmltv2vdr importer thread ended (pid=8959, tid=9576)

Cheers,
Ole

Actions
Copy link
 #6

Updated by OleS over 11 years ago

Hi again,

please ignore my previous post, compiled the wrong version...

Unfortunately, the new git version won't compile:

root@htpc:~/install/vdr-plugin-xmltv2vdr# make VDRDIR=/usr/include/vdr LIBDIR=/usr/lib/vdr/plugins
g++ -g -Wall -Woverloaded-virtual -Wno-parentheses -O2 -fPIC -c -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -D_XOPEN_SOURCE -DPLUGIN_NAME_I18N='"xmltv2vdr"' -I/usr/include/dvb-s2api-liplianin -I/usr/include/vdr/include -I/usr/include/libxml2 xmltv2vdr.cpp
xmltv2vdr.cpp: In Elementfunktion »bool cEPGHandler::check4proc(cEvent*, cTimer**, cEPGMapping**)«:
xmltv2vdr.cpp:431:49: Fehler: keine passende Funktion für Aufruf von »cTimers::GetMatch(cEvent*&, eTimerMatch*)«
xmltv2vdr.cpp:431:49: Anmerkung: Kandidaten sind:
/usr/include/vdr/include/vdr/timers.h:121:11: Anmerkung: cTimer* cTimers::GetMatch(time_t)
/usr/include/vdr/include/vdr/timers.h:121:11: Anmerkung: Kandidat erwartet 1 Argument, 2 angegeben
/usr/include/vdr/include/vdr/timers.h:122:11: Anmerkung: cTimer* cTimers::GetMatch(const cEvent*, int*)
/usr/include/vdr/include/vdr/timers.h:122:11: Anmerkung: keine bekannte Umwandlung für Argument 2 von »eTimerMatch*« nach »int*«
make: *** [xmltv2vdr.o] Fehler 1

Cheers,
Ole

Actions
Copy link
 #7

Updated by Joe_D over 11 years ago

  • % Done changed from 0 to 10

ok you are using an "outdated" vdr version ;)

please change

eTimerMatch TimerMatch = tmNone;
*timer=Timers.GetMatch(event,&TimerMatch)

into

#if VDRVERSNUM<=10732
int TimerMatch = tmNone;
#else
eTimerMatch TimerMatch = tmNone;
#endif
*timer=Timers.GetMatch(event,&TimerMatch)

in xmltv2vdr.cpp line 429ff

Actions
Copy link
 #8

Updated by OleS over 11 years ago

Hi,

on first glance, the new version seems ok:

Apr 19 13:45:29 htpc vdr: [25511] initializing plugin: xmltv2vdr (0.2.0pre): Importiert xmltv epg in den VDR
Apr 19 13:45:29 htpc vdr: [25511] starting plugin: xmltv2vdr
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using codeset 'UTF-8'
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using file '/srv/vdr/video.00/epg.db' for epg database
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using dir '/var/cache/vdr/epgimages' for epgimages (7)
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' added epgsource
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' reading source config
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is providing data through a pipe
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' updates data @00:00
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is needing a pin
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is providing pics
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' daysmax=14
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' reading plugin config
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' pin set
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' daysinadvance=3
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' using pics=0
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' weekdays=MTWTFSS
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' nextrun on Sat Apr 20 03:15:00 2013
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: 'epgdata2xmltv' is ready2parse
Apr 19 13:45:29 htpc vdr: [25511] xmltv2vdr: using sqlite v3.7.9
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr importer thread started (pid=25511, tid=25644)
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' epgdata2xmltv 3 'X' 0 13th-street.de 3sat.de animax.de anixe.de ard.de arte.de axntv.de das-vierte.de einsextra.de einsfestival.de einsmuxx.de fox-channel.de history-hd.de kabel1.de mgm.de orf1.at orf2.at prosieben.de rtl.de rtl2.de sat1.de sci-fi.de sixx.de sky-action.de sky-atlantic-hd.de sky-cinema-1.de sky-cinema-24.de sky-cinema-hits.de sky-cinema.de superrtl.de tele5.de tnt-film.de tnt-serie.de vox.de zdf.de zdfinfo.de zdfkultur.de zdfneo.de
Apr 19 13:45:33 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' executing epgsource
Apr 19 13:45:48 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' parsing output
Apr 19 13:45:51 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' processed 4181 xmltv events
Apr 19 13:45:51 htpc vdr: [25644] xmltv2vdr: 'epgdata2xmltv' importing from db

but 'importing from db' seems not. Importing this amount of data usually took
no more than 2-3 minutes. Now I'm up to 15 and still counting. Is this an expected
behaviour on your side?

Just to understand the process: importing means merging /srv/vdr/video.00/epg.db
into /var/cache/vdr/epg.data, or am I mistaken?

Cheers,
Ole

Actions
Copy link
 #9

Updated by OleS over 11 years ago

Update: after 20 minutes the import finished with no error.
Conclusion: it's slow but it works!

Thanks and cheers,
Ole

Actions
Copy link
 #10

Updated by Joe_D over 11 years ago

  • % Done changed from 10 to 90

I missed to remove a usleep in the code, please try the latest GIT!

Actions
Copy link
 #11

Updated by OleS over 11 years ago

Looking good now.

Thanks again,
Ole

Actions
Copy link
 #12

Updated by Joe_D over 11 years ago

  • Status changed from Assigned to Closed
  • % Done changed from 90 to 100
Actions
Copy link

Also available in: Atom PDF