added helper scripts for generation of eepg.equiv

updated documentation

6 files changed, 185 insertions, 28 deletions

diff --git a/HISTORY b/HISTORY
index a7b7626..f3ccd4d 100644
--- a/HISTORY
+++ b/HISTORY
@@ -15,7 +15,20 @@ Release 0.0.6pre:
 - Added setup option to display message after finish of writing epg
 - Added Dish Network and Bell ExpressVU support, thanks to the VDR patch, Mrgandalf and VDR User
 - Applied patch to Load Huffman dictionaries only on first use, thanks to Torsten Lang
-
+- Fixed update of equivalents for Freesat
+- Add category and genere to the description usable with epgsearch
+- Added EpgHandler for manipulation of the EPG for VDR > 1.7.26
+- Try to discard very long events that override more than one existing event
+- Equivalents stored in map for all equivalents for better handling
+- Move equivalence to separate file/class
+- Move utilities to separate files
+- Added cAddEventThread from VDR EPGFixer Plug-in to update equivalent channels in separate thread
+- Try to discard very long events that override more than one existing event
+- Use map instead of list in cAddEventThread to reduce sorting
+- Move cEIT2 in a separate file and try to use it in epghandler also
+- Drop unmanaged NOEPG support, there is a separate plugin for that
+- Drop unmanaged Disable Double EPG entry. EEPG tries to handle this anyway
+- Aditional fixes logged at http://projects.vdr-developer.org/git/vdr-plugin-eepg.git/
 
 Release 0.0.5:
 -changed TableId's so that Now/Next overwrites Nagra which overwrites MHW
diff --git a/README b/README
index 328ad04..20a9f89 100644
--- a/README
+++ b/README
@@ -1,23 +1,40 @@
 
-This is the Extended EPG (EEPG) plugin for VDR.
+This is the Extended EPG (EEPG) "plugin" for the Video Disk Recorder (VDR).
 
+Written by:                  Dingo35
 
-See the file COPYING for license information.
+Mainnained by:               Dimitar Petrovski <dimeptr@gmail.com>
+
+Project's homepage:          http://projects.vdr-developer.org/projects/plg-eepg
+
+Latest stable version available at:       
+    http://projects.vdr-developer.org/git/vdr-plugin-eepg.git/snapshot/vdr-plugin-eepg-master.tar.bz2
+Latest experimental version available at: 
+    http://projects.vdr-developer.org/git/vdr-plugin-eepg.git/snapshot/vdr-plugin-eepg-experimental.tar.bz2
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+See the file COPYING for more information.
 
 ----------------------------------------------------------------------
 
+Description:
+
 This plugin parses the extended (2 to 10 day) EPG data which is send by 
 providers on their portal channels. This EEPG data is transmitted in a 
 non-standard format on a non-standard PID.
 
 Currently the following EEPG formats are supported:
--Mediahighway 1 (CanaalDigitaalNL, CSat, Cyfra+)
--Mediahighway 2 (Digital+)
--Sky Italy
--Sky UK
--Freesat
--Premiere
--NagraGuide (CanaalDigitaalNL, only in test)
+* Mediahighway 1 (CanaalDigitaalNL, CSat, Cyfra+)
+* Mediahighway 2 (Digital+)
+* Sky Italy
+* Sky UK
+* Freesat
+* Premiere
+* NagraGuide (CanaalDigitaalNL, only in test)
+* NA Dish and BEV
 
 INSTALL
 Unpack, make and make plugins like any other plugin. Call like any other
@@ -29,6 +46,10 @@ E.g., CanaalDigitaalNL still sends Dorcel EEPG data on S19.2E, which can be used
 for Dorcel on S13.0E.
 Some sample eepg.equiv files are included.
 
+Scripts
+The provided scripts in the package can also help generate the eepg.equiv file.
+For more information see README in scripts/
+
 USE
 The plugin has no mainmenu entry, and only a few settings for Premiere. 
 When one of VDR's devices is tuned to a transponder (by tuning, a recording or 
@@ -36,7 +57,7 @@ an EPG scan triggered by VDR), the plugin scans all channels on the transponder
 for the supported EEPG formats. When all of the supported formats are detected, 
 all available EEPG data is loaded into the schedule of VDR.
 Additionally, the EEPG data is loaded into additional channels if they are specified 
-in the eepg.equiv file (not for Premiere or Freesat).
+in the eepg.equiv file (not for Premiere).
 Everything is done automatically and transparently, all is done concurrently
 with recordings and live viewings.
 Freesat and Premiere are loading their EEPG info continuously.
@@ -54,15 +75,15 @@ New, unknown EEPG transponders will be detected automatically (MHW1, MHW2, Frees
 have to be added (Sky uses different huffman decoding tables for every country).
 
 Currently known transponders that send EEPG data:
--Sky Italia       S13.0E:11881V (OpenTV)
--Sky UK           S28.2E:11778V (OpenTV)
--Cyfra+           S13.0E:10719V (Mediahighway 1)
--CSat             S19.2E:12363V (Mediahighway 1)
--Canal DigitaalNL S19.2E:12515H (Mediahighway 1, NagraGuide)
--Digital+         S19.2E:10847V (Mediahighway 2)
--Premiere         Sport Portal, Premiere Direkt Portal
--Freesat          all freesat channels
--Dish and BEV	  S119.0W:12472H S91.0W12224R
+* Sky Italia       S13.0E:11881V (OpenTV)
+* Sky UK           S28.2E:11778V (OpenTV)
+* Cyfra+           S13.0E:10719V (Mediahighway 1)
+* CSat             S19.2E:12363V (Mediahighway 1)
+* Canal DigitaalNL S19.2E:12515H (Mediahighway 1, NagraGuide)
+* Digital+         S19.2E:10847V (Mediahighway 2)
+* Premiere         Sport Portal, Premiere Direkt Portal
+* Freesat          all freesat channels
+* Dish and BEV	  S119.0W:12472H S91.0W12224R
 
 Please note that the NagraGuide broadcasts of Canal DigitaalNL are stil in betatest; 
 therefore its information is not always reliable. It seems that sometimes, when no summaries 
@@ -84,13 +105,14 @@ The plugin setup menu is only for Premiere protocol!
 
 THANKS
 This code is based on:
- -Premiere plugin (C) 2005-2007 Stefan Huelswitt <s.huelswitt@gmx.de>
- -mhwepg program  (C) 2002, 2003 Jean-Claude Repetto <mhwepg@repetto.org>
- -LoadEpg plugin written by Luca De Pieri <dpluca@libero.it>
- -Freesat patch written by dom /at/ suborbital.org.uk
- -Dish/BEV patch written by 
+* Premiere plugin (C) 2005-2007 Stefan Huelswitt <s.huelswitt@gmx.de>
+* mhwepg program  (C) 2002, 2003 Jean-Claude Repetto <mhwepg@repetto.org>
+* LoadEpg plugin written by Luca De Pieri <dpluca@libero.it>
+* Freesat patch written by dom /at/ suborbital.org.uk
+* Dish/BEV patch written by 
+* cAddEventThread from EPGFixer plugin by Matti Lehtimaki matti.lehtimaki /at/ gmail.com
 Thanks to mrgandalf, and the others who helped map NA eit.
-Thanks to VDR User for testing.
+Thanks to VDR User for testing and providing <> script.
 
 We wish to thank all authors for the great work they have been doing, decoding
 this EEPG data; this plugin tries to combine the best of all worlds.
@@ -106,7 +128,7 @@ information.
 
 
 KNOWN BUGS
--Equivalents file is not used for Freesat, Premiere.
+-Equivalents file is not used for Premiere.
 -On Sky Italy and Sky UK, a lot of "summaries not found" are reported. This is 
  because it is not (yet) known where the "summary available" flag is coded in the
  OpenTV protocol used, so all titles are assumed to have a summary available.
diff --git a/TODO b/TODO
index 888e3f2..50c9f51 100644
--- a/TODO
+++ b/TODO
@@ -1,4 +1,5 @@
 TODO/WISH:
+-Missing support for Freesat DVB-T
 -extend equivalents file with transponder SKIP (do not retrieve EEPG info) or 
  transponder ONCE (retrieve once a day, first possiblity after midnight). 
  TUNE (retrieve every time transponder gets tuned to) transponder CONT (retrieve continuously)
@@ -9,6 +10,5 @@ TODO/WISH:
 -stop premiere after having read all titles (from CONT to ONCE)
 -check version info for all providers
 -decode summary-available bit for OpenTV
--include DDEPG functionality
 -parallellize when not on FF card
 -NAGRA 2nd textstring is not used right now, find out when it is relevant...
diff --git a/scripts/README b/scripts/README
new file mode 100644
index 0000000..22e2835
--- /dev/null
+++ b/scripts/README
@@ -0,0 +1,28 @@
+extract_vdr_chan_ids.pl
+The script extract_vdr_chan_ids.pl can be used along with grep to extract the IDs 
+of required channel i.e.:
+
+USAGE:
+    extract_vdr_chan_ids.pl [/path/to/channels.conf]
+
+EXAMPES:
+    extract_vdr_chan_ids.pl /path/to/channels.conf | grep -i Chanel Name
+
+makeequiv.sh thanks to VDR User
+The makeequiv.sh bash script can be used to generate an eepg.equiv
+file based on SID matching. You may optionally require channel name
+matching as well.
+
+USAGE:
+    makeequiv.sh [channels.conf] [listsources|source to map epg from] <matchname>
+
+EXAMPES:
+    makequiv.sh /video/channels.conf listsources
+    * shows a list of all available sources in channels.conf
+    
+     makeequiv.sh /video/channels.conf S72.7W
+     * map epg from S72.7W channels based on SID match
+      
+     makeequiv.sh /video/channels.conf S72.7W matchname
+     * map epg from S72.7W channels based on SID and channel name match
+
diff --git a/scripts/extract_vdr_chan_ids.pl b/scripts/extract_vdr_chan_ids.pl
new file mode 100755
index 0000000..e40bbb9
--- /dev/null
+++ b/scripts/extract_vdr_chan_ids.pl
@@ -0,0 +1,38 @@
+#!/usr/bin/perl -w
+#
+# extract VDR channel ID's from channels conf
+#
+
+use strict;
+use File::Path;
+
+my $channelsConf = "../channels.conf";
+my $Usage = qq{
+Usage: $0 (default: ../channels.conf)
+       $0 /path/to/channels.conf
+};
+
+$channelsConf  = "@ARGV" if @ARGV;
+
+#my $file;
+open(MYINPUTFILE, "<$channelsConf") or die("Could not open $channelsConf" . $Usage);
+
+foreach (<MYINPUTFILE>)
+{
+    chomp;
+    if ( /^:.*/ )
+    {
+        print $_ . "\n";
+        next;
+    }
+
+    my($line) = $_;
+
+    my(@tokens) = split(":");
+
+    my($chanID) = "$tokens[3]-$tokens[10]-$tokens[11]-$tokens[9] $tokens[0]";
+
+    print $chanID . "\n"
+
+}
+
diff --git a/scripts/makeequiv.sh b/scripts/makeequiv.sh
new file mode 100755
index 0000000..1be3e80
--- /dev/null
+++ b/scripts/makeequiv.sh
@@ -0,0 +1,56 @@
+#!/bin/bash
+
+equiv_file="eepg.equiv"
+
+usage() {
+	echo
+	[[ "$@" ]] && echo -e "ABORTED! $@\n"
+	echo -e "usage: ${0##*/} [channels.conf] [listsources|source to map epg from] <matchname>"
+	echo -e "       * the 3rd argument is optional. if its \"matchname\", channel name matching is required for a positive match.\n"
+	exit
+}
+
+getsources() { awk -F: '{print $4}' "$1" |sort -V |uniq; }
+listsources() { getsources "$1"; exit; }
+
+(($# < 2)) && usage
+[[ -e "$1" ]] || usage "$1 not found"
+[[ "$2" == "listsources" ]] && listsources "$1"
+[[ $(getsources "$1" |grep "$2$") ]] || usage "$2 is not a source in $1"
+[[ "$3" == "matchname" ]] && match_name=1 || match_name=0
+[[ -e "$equiv_file" ]] && rm "$equiv_file"
+echo
+tput sc
+OLDIFS=$IFS; IFS=$'\n'
+for i in $(awk -F: -v var="$2" '$4==var' "$1"); do
+	matched=0
+	mapto_name=" ${i%%:*}"
+	mapto_sid=$(awk -F: '{print $10}' <<<"$i")
+	mapto_line=$(awk -F: '{print $4"-"$11"-"$12"-"$10"-"$13}' <<<"$i")
+	mapto_source=$(awk -F: '{print $4}' <<<"$i")
+	for j in $(awk -F: -v var1="$2" -v var2="$mapto_sid" '$4!=var1 && $10==var2' "$1"); do
+		mapfrom_source=$(awk -F: '{print $4}' <<<"$j")
+		mapfrom_line=$(awk -F: '{print $4"-"$11"-"$12"-"$10"-"$13}' <<<"$j")
+		matched=1
+		(($match_name)) && {
+			[[ " ${j%%:*}" == "$mapto_name" ]] || matched=0
+		} || unset mapto_name
+	done
+	(($matched)) && {
+		((matchcount++))
+		outline="$mapfrom_line $mapto_line$mapto_name"
+		echo "$outline" >>"$equiv_file"
+	} || {
+		((unmatchcount++))
+		array+=( "$i" )
+		unset outline
+	}
+	tput rc
+	tput el
+	echo "$outline"
+	echo -n "matched: $matchcount  -  unmatched: $unmatchcount"
+done
+IFS=$OLDIFS
+echo -e "\n"
+((${#array[@]} > 0)) && for i in "${array[@]}"; do echo "NO MATCH: $i"; done
+echo "wrote $equiv_file ($matchcount entries)."