Bug #244
closedttxtsubs can cause excessive cpu usage with some streams.
Added by mjl almost 15 years ago. Updated over 14 years ago.
100%
Description
Currently, ttxtsubs displays every page regardless of whether the page has new content or not, which depending on the stream, may be 100's (450 in one case here) of refreshes every second or so. This drives vdr's cpu usage through the roof (100% of one core), as well as causing an annoying flicker as the osd draw code tries to keep up.
Attached is a simple patch which discards duplicate frames. It also removes unnecessary ClearOSD() calls, as the ShowOSD() function deletes the osd itself as necessary. With the patch cpu usage remains stable even when 450 duplicate frames are decoded. The check should possibly be further up the pipe, but I don't know the code well enough.
Files
ttxtsubs-discard-dup-refreshes.diff (1.6 KB) ttxtsubs-discard-dup-refreshes.diff | mjl, 02/15/2010 05:30 AM | ||
ttxtsubs-downunder.patch (810 Bytes) ttxtsubs-downunder.patch | make only pages with the erase flag be shown | etobi, 02/19/2010 12:24 AM |
Updated by etobi almost 15 years ago
That's strange. Things seem to work differently down under :-)
I still have the recording you once uploaded and indeed, each subtitle page appears twice in the stream. The only difference between the two pages is in the page header. The page is sent once without the erase flag and once with the erase flag.
But these are only two times, the same text is displayed, which doesn't explain the problem you described. Can you please make another short recording, where there are 100's of refreshes?
Updated by mjl almost 15 years ago
etobi wrote:
That's strange. Things seem to work differently down under :-)
Indeed. Even the water flows in a different direction as it goes down the drain :) I'm uploading an Advert to mjlmedia.com/vdr/ - it should be there in the next half-hour or so... it's the only recording I have with subs crazy subs at the moment, but it does show 50->210 dupe frames, and drives vdr to around 30% cpu utilisation without the hack.
The most problematic channel is not showing subs with the current program, but if you need a longer example I'll see what I can do.
Updated by mjl almost 15 years ago
etobi wrote:
I still have the recording you once uploaded and indeed, each subtitle page appears twice in the stream. The only difference between the two pages is in the page header. The page is sent once without the erase flag and once with the erase flag.
Does the plugin currently pay any attention to the erase flag? Looking at the code, it seems to delete then recreate _osd on every call to ShowOSD()
Updated by mjl almost 15 years ago
ok. the recording and its peripheral files are now uploaded. btw, I'm not using a fullfeatured card, but rather vdr-xine, if that makes any difference to cpu utilisation.
Updated by etobi almost 15 years ago
Ok, I've downloaded it. I'll take a look at it tomorrow.
Updated by etobi almost 15 years ago
- File ttxtsubs-downunder.patch ttxtsubs-downunder.patch added
Yes, I can see the problem.
On a German channel, the subtitles are sent 1 page per subtitle text with the erase flag set.
The snippet from the TCM recording you once uploaded has two pages per text. One with the erase flag set and one without.
The last snippet you uploaded is totally crazy and contains dozens of pages without the erase flag and then a single page with the erase flag.
The attached patch will make only pages with the erase flag be shown. This should solve your problem, but I still don't understand it. Either your broadcaster has a buggy teletext encoder or I'm missing something in the ETS specs.
Maybe they are trying to send subliminal messages via teletext :-)
Updated by mjl almost 15 years ago
Unfortunately, the patch seems to break the majority of subs. Subs that are broadcast "live" are updated word-by-word and don't display at all (the page is never erased, just updated by the looks), and many others fail :( The good news is that the snippet that I sent is fixed :) ... I'll upload a couple more sample streams, perhaps they will provide some further insight...
Updated by mjl almost 15 years ago
Two new samples are now available in the usual place. One with live subs, the other loses around 70% of the subtitles if the above patch is used.
Updated by mjl almost 15 years ago
and a second sample of the crazy subs, on the off-chance that it's a slightly different cause.
Updated by etobi almost 15 years ago
Thanks! I'll check your samples over the weekend.
Updated by mjl over 14 years ago
Sorry to pester you, but any further updates on this? If you want, you can close this bug, as I seem to be the only one with these streams and I'm content to simply use the patch I provided to work around those.
Updated by etobi over 14 years ago
I checked all three recordings you uploaded and I can't find anything in the teletext specs, that explains this. I've also checked your recordings with VLC, where two of them work, but with the live recording the subtitles go mad. The other two recordings work, because VLC uses the same workaround as your patch and checks for duplicate subtitles. As far as I know, no such problem exists in Finland or Germany, so I think your broadcaster uses a buggy teletext encoder. Is this happening on all channels?
If I don't find another solution, I'll add a setup option to check for duplicate subtitles - we already have a French workaround, so why not add an Australian one too :-)
Updated by mjl over 14 years ago
All terrestrial and satellite broadcasters in AU seem to have the same set of issues.
I briefly scanned through EN300706 and found nothing either, except for a brief passage that recommends checking for duplicate MIPs. I haven't yet read through the code to determine whether ttxtsubs does that, nor do I know if its relevant. I found nothing that explains what the providers are doing with 'live' subs. It's my experience (when writing the xine dvb subtitle decoder) that broadcasters adhere to the spec to the letter, except when they want to 'improve' it :)
Updated by etobi over 14 years ago
What I have missed so far is, that the ShowOSD method is also triggered by the empty pages following a subtitle text. I thought the subtitle page would be sent with the same text over and over again (which was the case in one of the samples), but in the other samples the text appears only twice (first time with the erase flag set, then without) and then a lot of empty pages (without the erase flag) follow. These pages should be ignored.
The latest Git revision will do this. ShowOSD will only be triggered, if the page content changes or if a page with the erase flag was received.
This should fix the CPU load problem without introducing a new setup option. It will not fix your live sample, this still doesn't make sense to me and even VLC only shows chaotic subtitles.
Please test the latest Git snapshot and let me know, if it works.
Updated by Anonymous over 14 years ago
- Status changed from New to Closed
- % Done changed from 0 to 100
Applied in changeset 7c1f03fc2c90fa491eedfdf2a8932cb8ac1db659 and 29a737636fb3791fb5d38b5584b951add7c58106.
Updated by mjl over 14 years ago
Yes, that seems to work for the streams I have available at the moment. Thanks for persevering.
What do you mean by 'chaotic subtitles'? I've worked around the issue of flicker by only destroying and recreating _osd if its vertical size changes. Is that what you meant?
Updated by etobi over 14 years ago
With "chaotic subtitles" I meant your sample with the live captioned program. In thise sample,
a single
a single text
a single text line
a single text line in a
a single text line in a subtitle page
:-)...gets updated as the captioner types the text. There are at least three issues with this:
- centered text becomes hard to read because every update realigns the text line - the text should be left-aligned here
- When displaying two rows, it seems some old text isn't correctly deleted
- It seems the subtitle colors can change within a line. When adding color support I explicitly assumed, there's only one color per text line to make things easier.
Updated by mjl over 14 years ago
heh, actually, I don't find it too bad :) The changing subtitle colors are a little annoying at times, but I can deal with it. With the vdr-xine softdevice plugin the subs flicker like crazy when displaying these live subs, but I've hacked around that particular problem. If you feel like resolving any of the issues you mention I'd be happy to assist with testing, but at the moment, I'm 'OK' with the current state of affairs, and am happy to stop bugging you for now :)
Updated by etobi over 14 years ago
Ok, then I'll address this problem later. I'll prepare a release and then start refactoring the code before adding any new features or fixes.