About¶

vdr-convert is a command line tool to accurately transcode VDR1.x and VDR2.x recordings, including all valid streams - video, audio (including AC3/DTS 5.1), Audio Description (AD), and DVB subtitles - into more compressed and accessible formats, while maintaining perceived quality with good compatibility. There are 2 basic modes: a) for keeping recordings inside VDR, and b) for using with other media players.

H264 and AAC are the default codecs for the main streams, but H265 is available, together with most common multimedia file formats for use in single-use/batch/podcast modes. Standard Definition (SD) recordings are reduced to anywhere from 35% - 90% of original size depending on content and compression settings. The typical transcoded file is 60-70% of original size over a range of SD recording types. The better and less noisy the content, the better the compression, e.g. current SD output from the BBC is often compressing to 35-45% of original size, typ. 450-800M/hr (in keep mode, H264 CRF=21). Lower quality transmissions, from lower bitrate broadcasters (which blurs the video slightly), results in less compression, and counter-intuitively a larger compressed file when tested on identical material - the exact same content can be as much as twice as large.

When keeping files for use inside VDR, it is assumed that the user has VDR2.x with an HD capable setup, and so can play anything using the above codecs. A number of size, duration and compatibility tests are performed before (optionally) deleting the originals. When keeping VDR1.x recordings, a new directory is made in VDR2.x format and the metadata files renamed, amended with new tags, and moved over.

Rationale¶

Disk space is cheap - why bother?

• Disk space isn't quite so cheap or easy to manage when it's in a RAID array, a phone or tablet or other mobile device with limited storage capacity.
   
• A key driver was a desire to recover a substantial archive of rarely transmitted shows recorded over 8+ years, mostly using VDR 1.6. The older .vdr private file format contains subtitles that are encoded uniquely, so they won't show on anything other than certain legacy VDR configurations; these recordings can be difficult to view/transcode for 3rd party players (multiple files, file format not always properly recognised, accessibility features missing).
   
• The need for a semi-automatic compression/FTP upload/podcast facility creating a more compact copy for offline entertainment - when working abroad, or travelling with a mobile with limited or no Internet access.

There are many other solutions (like the very original vdrconvert [http://vdrconvert.vdr-portal.de] after which this project is named) and simpler scripts/programs out there, but most do the much simpler job of single video/audio stream conversion without VDR 1 > 2 format modification, subtitle extraction and conversion, validity checks, and indexing. Additionally, vdr-convert applies a number of carefully selected x264/x265 and mpegts multiplexer optimisations appropriate to TV recordings, to make sure that the recordings look as good as possible, streams are well ordered and labelled. Usability, such as good skip forward/backwards performance, has also been considered (default ffmpeg settings are not ideal). A number of optimisations are also carried out on podcast conversions, including sample rate conversion to 44.1kHz for wide player compatibility.

More importantly, due to the complexity of the steps required, in testing no other conversion method could successfully recover all valid streams (and deal with empty streams), particularly DVB subtitles in VDR 1.x files with their correct colours, clean fonts, properly synchronised for use by current VDR (2.2+) AND other players.

Usage¶

vdr-convert can be used in 3 main ways:

1. "Keep" mode (-k) - after a recording is made - in exactly the same way as noad, markad and similar tools - run by VDR automatically. The recording files are compressed, re-indexed and recognised by VDR, but are significantly smaller. If already compressed with the target compression type (e.g. H264), the recording is not modified. The author uses this on all recordings with lifetimes > 7 days
    
2. In its original "single-use" mode with approximately doubled compression and conversion speed, together with optional FTP integration to upload converted files to a remote FTP server. Unlike the "keep" mode, this will also further compress H264/H265 streams. By default, the output is MPEGTS (.ts) with AAC as main audio stream (as in keep mode), but most other common formats are supported - details below. In this mode it will also create an edit-decision-list(.edl) if a marks file is present to facilitate commercial skipping in players such as mplayer.
    
   A flavour of this mode is the Podcast-mode which outputs transcoded video or audio files in hierarchical directories organised by Title, providing more than double the compression for audio files with HE-AAC v2, plus inserting comprehensive metadata tags and providing indexing hooks for media servers such as a Squeezeserver, and common media players.
    
3. In a batch, to convert as many or as few recordings as required, either for keeping in the VDR environment or for single-use.

There is no UI for batch mode as vdr-convert is often used just once in this mode. A text "todo" file is created, and then a small script called batch executes vdr-convert on each file. This allows the user to batch up as many or few recordings as preferred, filter and choose the order.

vdr-convert always outputs exactly ONE multimedia file as this was its original design for convenience.

A file for single use is named using available VDR recording metadata: TITLE-SUBTITLE-EP-DATE-DOW_TIME-SD/HD/AUDIO.ext.
Example: Rick-Stein's-Long-Weekends-Palermo-EP-0.9-10.-2017-09-23-Sat_12-00-SD.mkv

See Howto and Options for full detail

Player and file compatibility¶

MPEGTS (.ts) is used as the default output container, as it is the native format now used by VDR, and the most robust one supporting all streams and player combinations. However other containers such as MKV are around 5-10% smaller (not having the mpegts packet overhead) and have other advantages such as supporting metadata tags.

The following containers/formats are supported with the -f option in single-use and batch modes: avi, mkv, mp4, mov, m2ts and ts, and for radio/audio aac, m4a, mkv, mp4, mp3 and ts. Others may work too, depending on your ffmpeg build, but are not tested.

H264, H265 and copy (MPEG2/H262 or higher) video codecs are supported using the -v and -m options, and native AAC, Fraunhofer AAC or libmp3lame(MP3) are supported as audio codecs using the -a option

Tested video players and platforms are shown in the following table for H264 and H265 video codecs.
Note that some containers have design limitations; mp4 and avi don't support DVB subtitles, ts and m2ts don't support useful metadata

In summary, some players are good for almost all formats, and if you don't need subtitles or automatic aspect ratio switching for 4:3 or other aspect ratios, most players will play most formats. The author mainly uses VDR 2.2 with Kodi/Raspberry Pi frontends. For use outside vdr/Kodi, the author recommends .mkv for best balance of size, stream support, metadata and compatibility, and default .ts for best overall compatibility.

Note that features of tested players definitely varies by version, bugs get fixed, others appear. Tests above may only be valid for the versions shown.

Quality¶

See Conversion and Workflow for a discussion about conversion parameters, quality and file sizes

Software required¶

If keeping recordings for use inside a VDR environment, you need VDR 2.2x installed on the conversion machine, otherwise not.

All recordings¶

• System running Linux/bash shell
• Core Linux utilities such as nice, bc, timeout etc
• FFMPEG 3.x or 4.x, built with any required libraries (e.g. libx264, libx265, libfdk_aac, libsoxr). Libx264 is a minimum.

Additionally for VDR 1.x recordings¶

• Modified GENINDEX (0.2 or later) to convert DVB subtitles to standard ETSI EN300743 format, remove initial short segment sequences that currently prevent ffmpeg reliably detecting subtitle streams, and remove substream headers from AC3/DTS streams for ffmpeg compatibility.
  (GENINDEX is included in this project in the repository: source:genindex)
• MPLEX13818 from http://www.scara.com/~schirmer/o/mplex13818 to reliably convert recordings into an mpegts container. ffmpeg doesn't handle dvbsubs in a PES stream (.vdr native format), and sometimes fails to recognise .vdr files correctly.

Optionally¶

• NCFTP client if you want to upload files after conversion

See the source:Readme.txt for details of how to build and install the software components from the repository

Patching¶

• You should consider patching ffmpeg per source:Readme.txt, because stock ffmpeg is intolerant of subtitles streams with any DTS timestamp errors. If you have broken recordings - e.g where there was reception interference during recording, or DTS timestamp errors, stock ffmpeg will exit. It should be noted that some second tier broadcasters regularly transmit programmes with DTS problems, probably caused by inserting commercials, so depending on your recordings, patching may be essential. source:FFMPEG patches
• VDR benefits from a couple of minor patches (source:VDR patches) a) to suppress warnings when reindexing, and, at the time of writing, b) to allow re-indexing of audio-only recordings
• See the source:Readme.txt for full details

Issues / testing¶

vdr-convert has been run successfully and refined on 2000+ recordings from a variety of UK DVB-T and DVB-T2 (HD 1080i) channels, German DVB-S channels with MP2 and AC3 audio streams, made using VDR versions 1.3x - 1.6x and 2.2 over a period of 10 years. Please note however that the author has a headless VDR server, so native VDR playback onscreen has not yet been tested. Aside from any compatibility issues shown above, the following minor issues exist

• Simultaneous AC3/DTS audio and DVB subtitles in VDR1.x recordings have yet to be tested, but are logically expected to work. Both AC3 and subtitles have been independently and extensively tested on a variety of recordings. Please contact the author if you have suitable samples !

• The languages for multiple audio and subs streams of the same type may not be tagged correctly in the output files because ffmpeg does not specify the order that it detects the streams which have been observed varying between files, and so will not match VDR. Currently the language of the first matching stream type (stereo, mono, AD, 5.1, subs etc) is used for all streams of that type. This will not necessarily be correct for recordings with, for example, both German and English stereo streams.

• Broken VDR1.x recordings comprising more than 1 file do not convert well to MP4 due to timestamp issues - use another format.