Howto¶
To use after automatically after a recording¶
Use the provided vdr-auto script, and edit your VDR command line to include
-r /usr/local/bin/vdr-auto
Recordings are automatically converted in "keep" mode when the recording ends, if the programmed "lifetime" is >= value set on the command line with -l option (default 8 days). vdr-auto includes a call to noad
To use as a batch (batch.sh)¶
Create a todo.txt file as follows at whatever level you like in your VDR recording file hierarchy.
find ./ -type d | grep -i '.rec$' |sort > ./todo.txt
Edit the resulting file to suit, and then run it as a batch in the directory containing the todo.txt.
nohup allows the process to continue when you logout.
nohup batch.sh -k -d &
Convert a recording for single use¶
A normal conversion for use on a portable player in Matroska format
vdr-convert -f mkv -i "recording.rec"
Example of output: More or Less-EP-EP013183250028-2016-07-29-Fri_16-30-AUDIO.mkv
In this example, a radio recording is converted to mp3. vdr-convert forces the use of the libmp3lame codec if mp3 format is selected
vdr-convert -f mp3 -i "recording.rec"
Podcast mode¶
Here a radio recording is converted to a tagged, high quality, highly compressed HE-AAC v2 audio podcast, with original files deleted after successful conversion. The file is put in a subdirectory using the recording title below
vdr-convert -d --podcast /mnt/lvm/Radio -i "recording.rec"
Example of output: /mnt/lvm/Radio/More or Less/More or Less-EP-EP013183250031-2018-08-31-Fri_16-30.m4a
See Podcast-mode for more description
A full description of Options is provided.
Logging¶
The script provides summary information to your syslog (example below), and detailed ffmpeg output to /var/log/vdr-convert
Aug 22 09:05:49 ha-server vdr-convert: Processing recording "./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec", (files=2) keep=1, delete=1, combine=1, subs=1, extras=-tune film Aug 22 09:06:35 ha-server vdr-convert: Processing ./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec/recording.orig file 1, size 2669M... Aug 22 09:07:23 ha-server vdr-convert: ./From_Dusk_till_Dawn/2009-01-06.23.40.50.99.rec/recording.orig genindex extracted 4178 kb subtitle data Aug 22 09:11:24 ha-server vdr-convert: Executing nice -n 19 /opt/data/develop/FFmpeg/ffmpeg -y ... Aug 22 10:22:21 ha-server vdr-convert: Executing nice -n 19 /opt/data/develop/FFmpeg/ffmpeg -y ... Aug 22 10:22:52 ha-server vdr-convert: Created/replaced .ts and indexed From Dusk till Dawn-2009-01-06-Tue_23-40-SD (2669M to 1612M (60.4%))
Usage hints¶
Batch process files with similar requirements. The --extras (-e) option allows you to pass a set of arbitrary parameters to ffmpeg / codec libraries to optimise certain conversion types.
For example if you have a batch of films, you may want to use the -e "-tune film" flag to use "film" tuning with x264. An example command might be:
nohup batch.sh -k -d -e "-tune film" &
As an example of the use of multiple parameters, the author once used "-tune film -aspect 16:9" where the broadcaster had transmitted the first part of the film incorrectly squashed to 4:3 format - when corrected the elongated logo proved the error!
Default settings are quite conservative, aiming for quality over size reduction, results are indistinguishable in terms of quality - at least to the author's eyes and ears. If you want to save more space, test by increasing the -q value until you notice a drop in quality, note that the default value for keeping is significantly better than ffmpeg default of 23 (which is the value used in single-use mode)
vdr-convert -k -q 23 -i "recording.rec"
Deinterlacing¶
x264 produces slightly more compression on deinterlaced content, but most broadcast TV content is still interlaced today, even HD. Deinterlacing loses some temporal information, and however good, will produce some artefacts - there is a trade-off between saving space (a few percent), and visual quality. The particular issue with SD material is that it is likely to be upscaled to a larger screen than it's native resolution, any deinterlacing done during conversion at the recording's native resolution cannot take advantage of the ability of a modern TV or media player to interpolate/smooth several lines at it's higher resolution, producing significantly better results. Yadif deinterlacing by vdr-convert/ffmpeg although good, will often lead to more visible artefacts on moving parts of scenes, particularly where there are sharp near-horizontal or vertical lines, moving foliage or on sports events.
By default vdr-convert therefore maintains the same interlacing as the source. Any deinterlacing is best left to the TV or player at it's higher display resolution. This is also more future-proof as once deinterlaced, there is no viable way back. The TV / player (usually automatic), or end-user can decide what type of deinterlacing to apply for best results.
vdr-convert provides an option to enable the yadif deinterlacer if required. The most efficient conversion occurs with y=0. y=1 makes the yadif deinterlacer create 1 frame per field which can improve perceived quality on noisy or fast moving recordings, with perhaps a 20% size penalty.
H265 (HEVC)¶
x265 has been developed from x264 and shares many features, and is supported as long as ffmpeg has been built with x265 library support. It can produce files of half the size of H264, and in the author's opinion, the quality has recently improved(2018) and nearly matches H264's output. It is constantly improving, so worth trying with each new release, meanwhile it may be better just for HD content with a fixed aspect ratio.
Some conservative settings have been provided for x265, and it's probably best used in the single-use mode if maximum compression is needed.
Important notes:¶
- Conversion with the slow preset is extremely slow, esp. on HD content. It could be less than 10% of real time, even with a reasonable CPU.
- H265 is currently not useful for any content that has aspect ratio changes, because of ffmpeg/libx265 bug https://trac.ffmpeg.org/ticket/5959. The aspect ratio remains the same from the beginning of the recording, unlike H264 which changes aspect as content aspect changes. This is particularly noticeable on recordings of 4:3 material with 16:9 commercials inserted. Some broadcasters now pad 4:3 material to 16:9 with black borders to mask this issue.
VDR1.x recordings¶
VDR1.x recordings are automatically detected and can usually best be transcoded using the -c option to combine/concatenate files before conversion at the cost of an additional temporary copy of the whole recording during conversion where multiple files are present. This produces a seamless transition at any joins, avoids ffmpeg potentially recognising streams differently between multiple files 001.vdr, 002.vdr..., and just occasionally where a transition between files coincides with a broadcast program change (obviously a rare occurrence). This is a highly recommended option for these recordings, and can safely be used on VDR2.x .ts recordings also. The reason -c is not switched on by default is that broken recordings (interference etc) can fail when combined with -c
Note that VDR1.x conversions are disk intensive when they start. Several copy/conversion/test stages were found essential to achieve reliable conversions with synchronised streams, as documented in Workflow. Reducing the processing stages during development by, for example, having ffmpeg create main and subs files simultaneously lead to failures. Most processes are nice'd to the lowest priority, but ffmpeg in particular will consume all spare CPU resource. In practice this does not interfere noticeably with normal operation of a server - the author runs the script 24x7 on a home automation server that has many real-time processes running, such as analog CCTV capture, and VDR itself of course.
Remember to patch and rebuild your own copy of ffmpeg per the readme if you are having issues with conversions exiting due to noise or subtitle issues. You can test using the -n option to disable subtitles to verify if the cause is subs.
Keeping streams as MPEG2 / move to VDR2.x ts format and directory structure¶
If you just want to recover subtitles and/or AC3 audio and/or move to the current VDR2.x directory structure with .ts files that can also be played on a wide range of players, Kodi, MPC-HC, Mplayer and VLC etc, you can use this command which will take only a minute or two to convert each recording. It (losslessly) copies the MPEG2 video and audio streams and muxes in the recovered DVB subtitles and AC3 audio if available. In this case the recording will be about 3% larger than the original due to mpegts container overhead.
vdr-convert -c -k -m -d -i "recording.rec"
Troublesome recordings¶
vdr-convert does a number of tests on the transcoded recording; including errors reported by ffmpeg and other tools, file sizes, duration, and VDR reindex when using the -k option. If any fail, it exits, leaving the transcoded file in the recording directory, as well as originals. However, you can normally expect a 98%+ success rate.
Subtitles¶
Subtitles can cause problems for ffmpeg, often seen as early exit and shortened files (vdr-convert usually detects and warns you of errors or size issues). Check your log for the word "fail" and/or configure email notification. vdr-convert tests subs before conversion, and uses several ffmpeg timestamp options and other strategies to workaround common issues but cannot always manage broken files, which are almost always caused by interference, breaks in the recording, or very occasionally the broadcaster outside spec. In the author's experience significantly fewer than 1% of recordings might suffer this problem. If you have recordings with major DTS timestamp problems, you have a few options:
- Use -m option to copy MPEG2 streams, but still extract subtitles (and AC3/DTS if present), with the ~3% size penalty - this usually works unless the problems are severe. A useful trick, especially if scaling/fading/cutting the recording, is to convert the recording temporarily using the method described above, which is very quick, then do any further transcoding with other options as required afterwards. ffmpeg is usually significantly better behaved this way.
- Use the -n option and do not attempt to transcode subtitles at all.
- Finally it is usually possible to manually recover and recombine subs from a broken recording, the conversion order is different for VDR1 and 2, as follows...
VDR1
First, transcode using the -c and -n options (do not use -d option), which almost always works OK, leaving you with 00001.ts containing just complete transcoded AV streams. Then use the -j option to create just a subs.ts in the temp directory below the recording.
VDR2
First, use the -j option to quickly create just a subs.ts in the recording directory. Then transcode using the -c and -n options (no -d option), which almost always works OK, leaving you with 00001.ts containing just complete transcoded AV streams.
VDR1 and VDR2
Now manually combine the AV file 00001.ts and subs.ts (quick), keeping originals, remember to use your patched ffmpeg, otherwise it's likely to fail again:/path-to-your-ffmpeg -y -probesize 250M -analyzeduration 600M -copytb 1 -i 00001.ts -i subs.ts -c:s copy -c:v copy -c:a copy -map 0 -map 1 temp.ts
Watch temp.ts (MPC-HC or SMPlayer with mpv engine recommended), subs timing is likely to be way out of sync with the audio/video, several seconds to several minutes, so manually time the difference between subs and audio/video (e.g. xx seconds). Apply this time difference in a quick remux:/path-to-your-ffmpeg -y -probesize 250M -analyzeduration 600M -copytb 1 -i 00001.ts -itsoffset 00:00:xx -i subs.ts -c:s copy -c:v copy -c:a copy -map 0 -map 1 temp.ts
Re-check temp.ts for subs sync, replace as 00001.ts when correct.
The author has successfully used this method on a handful of files which failed during normal conversion.
Other issues¶
If you have breaks in recordings leaving short fragments, perhaps less than 1MB, the -c (combine) option is useful to join them up (ffmpeg doesn't like very short fragments because its stream probing routines need a significant amount of material to correctly identify all stream types). If the fragments are at the start or end of the recording and not part of the programme, it's safer to just delete them before processing.
If a recording fails, you should first view the output file using one of the listed compatible players, as it may in fact be OK, having failed tests due to, for example, interference effects or breaks during the original recording which can alter apparent duration. If OK, then you can rename the file and delete the originals. Otherwise use one of the strategies above.
Very occasionally the transcoded recording can be larger than the original - this is notified as a fail, and originals retained. Extremely grainy films or film-based programmes can suffer this problem. In this case you can increase the compression with the -q option, revert to the original, or if VDR1.x recording, perhaps best to use the -m option to recover the subtitles but retain the original MPEG2 A/V streams.
ffmpeg has bugs! The author has logged several, https://trac.ffmpeg.org/ticket/5617 and https://trac.ffmpeg.org/ticket/5796 are most relevant at the time of writing. A very small percentage (<0.5%) of tested recordings simply can't be converted with subs. The ffmpeg patch in this project relates to #5617 and helps with poor recordings