Two DVB-T recordings with problems after processing by TS-Doctor

[charlesc]

TS-Doctor is a great program, thanks for your continued work on it.

I pass all my DVB-T transport stream recordings through it before top-and-tailing them in VideoReDo Plus.  I've been following this same work-flow for some years now, and it works well.  While editing a couple of recordings this afternoon I came across two of them that had some issues initially just seeking, and I noticed that the time length being shown in VRD Plus was way off.  It showed about 10 hours, not 1 hour.  And Daum PotPlayer showed the same 10 hours.  So does MediaInfo.

I then ran those files through VRD Plus's Quick Stream Fix, and it corrected the file.  They now reported about 1 hour length, and behaved correctly when seeking back and forth in the VRD timeline.

I've kept those files, and the TS-Doctor log of them when they ran through.  I was wondering it they might be of use to you to have a look at in case you can see what the issue was that didn't seem to be fixed by TS-Doctor.  The only problem is they are large files (about 2.7GB), but I can put them up somewhere if you need them.

This is the detail for one of the files (MediaInfo).

General
ID                                       : 1538 (0x602)
Complete name                            : Q:\PROCESS\KEEP for TS-Doctor\prob Frasier ELEVEN Wed (2015-05-20 14-40)_TSD.ts
Format                                   : MPEG-TS
File size                                : 2.79 GiB
Duration                                 : 10h 8mn
Overall bit rate mode                    : Variable
Overall bit rate                         : 655 Kbps

Video
ID                                       : 516 (0x204)
Menu ID                                  : 1576 (0x628)
Format                                   : MPEG Video
Format version                           : Version 2
Format profile                           : Main@Main
Format settings, BVOP                    : Yes
Format settings, Matrix                  : Custom
Format settings, GOP                     : Variable
Codec ID                                 : 2
Duration                                 : 10h 8mn
Bit rate mode                            : Variable
Bit rate                                 : 429 Kbps
Maximum bit rate                         : 15.0 Mbps
Width                                    : 720 pixels
Height                                   : 576 pixels
Display aspect ratio                     : 16:9
Active Format Description                : Full frame 16:9 image
Frame rate                               : 25.000 fps
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Top Field First
Compression mode                         : Lossy
Bits/(Pixel*Frame)                       : 0.041
Stream size                              : 1.82 GiB (65%)

Audio
ID                                       : 681 (0x2A9)
Menu ID                                  : 1576 (0x628)
Format                                   : MPEG Audio
Format version                           : Version 1
Format profile                           : Layer 2
Mode                                     : Joint stereo
Mode extension                           : MS Stereo
Codec ID                                 : 4
Duration                                 : 10h 8mn
Bit rate mode                            : Constant
Bit rate                                 : 192 Kbps
Maximum bit rate                         : 197 Kbps
Channel(s)                               : 2 channels
Sampling rate                            : 48.0 KHz
Compression mode                         : Lossy
Delay relative to video                  : -330ms
Stream size                              : 835 MiB (29%)
Language                                 : English

Menu
ID                                       : 263 (0x107)
Menu ID                                  : 1576 (0x628)
List                                     : 516 (0x204) (MPEG Video) / 681 (0x2A9) (MPEG Audio, English)
Language                                 :  / English
Maximum bit rate                         : 6736400

[Cypheros]

Yes, I would like to look into the problem. I sent you a PM where you can upload sample files.

[charlesc]

I've uploaded the two log files, and am in the process of uploading ONE of the large files.  This is an 8 hour process, about 2.7GB file.  If you need the second file, I might queue that up later.
I'm not sure if you got my replies to your PM. I was having issues getting Filezilla to work on default settings - I couldn't see folders on your server.  Fixed it by finding a Google post saying to turn of security on the transfer, ie. make it 'unsecured'.  That worked.

I've found another three or four files with the same problem.  Interestingly they all come from the same TV transmission (tranmitter, multiplex), so there may be something going on there.  It is TCN 10 here in Australia, one of their multiplexes is named 'ELEVEN'.  That is the one that had these replays of Frasier.

Note that I am editing a lot of these Frasier files at the moment, that may be one reason I am finding the Frasier/ELEVEN related problem.  It may be there on other recordings that I don't know about yet from other TV stations.  Hopefully with a file you may be able to see something in the stream.

[charlesc]

The file transfer seems to have aborted at about 80%, and re-started.  That was after about 9 hours!  Transfer speeds less than 100 KiB/s.
It restarted and was showing 0.2% at about 25 KiB/s speeds.  So I canceled.


Are most of the uploads around that speed?  I can try again overnight, but it may get most of the way there again and fail.

It left a part file of more than 2GB.  That probably won't be useful I suppose.

[Cypheros]

No, network connection is 16MBit/s.

[Cypheros]

OK, checked the 2 GB part you already uploaded. There is an error that needs to be fixed before you can do anything else with the file.

Code Auswählen Erweitern

TS Warning: PCR timing for PID 0081 last PCR:00:07:31.010 current PCR:08:46:03.919 at packet 00172556 TimeBase:2840752738 PCRExtension:185
Start patching PCR with additional timer offset -08:38:32.877 at packet 0000000000172556 -> 0000000000172529

You should get this message:
[attachimg=1]

If you click on "Yes" the file will be fixed, the error is gone and the time is correct:
[attachimg=2]

[Djfe]

normally you should be able to restart the uploads just fine (filezilla will just continue where it stopped)

[charlesc]

@Djfe   I thought a resume should be possible, but it didn't happen.  No problem, the second file seems to be at over 90% transfer now overnight.


@Cypheros
That's interesting what you've found.  I'll look into it now.
All these files are processed by TS-Doctor as batch files, and most of the dialogs are 'silenced'.  I'll check a file manually, and check how the answer to the dialog is set.
To be honest, as I put so many of these files through, the error checking that I tend to do most of the time is limited now to looking at the size of the error log text file in Windows Explorer.  If they are all much the same (like around 27KB) then I assume not too many errors have occurred, and I leave the rest to VRD Suite to handle the file.

The second file is hopefully about to complete 100%, but it looks like you have identified the likely issue.

I'm just not sure how to have this handled in a batch mode.

EDIT:  I just checked the settings of 'Invalid timer jump discovered. Create new file now' and it was set to No.  I set it to Yes, and ran a single file batch through.  TS-Doctor stopped the batch and asked if I wanted 'Confirm:  Open fixed file'.  If I said Yes, it completes and the file is corrected (displays the correct 1 hour length).

But the batch processing is stopped by this.  I also have No set to 'Program switching detected.  Create new file'.

In my situation, I would probably be happy for TS-Doctor to just do what it thinks is best, automatically.  I don't think I can recall ever having a file made useless by the batch processing.

[charlesc]

I just thought I'd try running the problem file through manually, in case the batch mode silenced any dialogs.  This is because I didn't see that confirmation dialog you showed the picture of, the one where it says 'remember this setting'.

My typical batch files btw are like this
tsdoctor.exe Q:\PROCESS\*.ts Autofix F:\PROCESS\fixedbatch\

With the 'Invalid timer jump discovered, create corrected file' set to YES, I manually selected the file with TS-Doctor and ran it through.  The only dialog I saw was 'Open corrected file'.  Saying Yes produced a corrected file.

[Djfe]

since you now checked "yes" as the default answer for that dialogue it also doesn't show up in single runs

batch runs are implemented rudimentary
they are pretty much single runs attached to each other (that's why there are sometimes bugs, like "do you want to open the corrected file?", which logically shouldn't show up)

overall the batch run works fine though I personally don't use it for normal recordings that I want to archive

I have used it once on a big folder (with default settings for all dialogues (except programm switching, because I wanted to decide there manually for obvious reasons -> would be nice to tell the doctor to produce two recordings there or simply leaving the switching in, but that's not possible)) which contained all recordings from my settopbox that I only want to watch once anyways (I wanted to make some space by removing nalu fillers and stuff like that)