MKVmerge warning on file produced by TS Doctor

[Sjakko]

Dear TSDoctor developer,

I'm using TS Doctor to repair and cut my DVB recordings but I encountered an issue.

On one of my DVB recordings something went wrong with decryption. The result was that I ended up with my recording split into 2 TS files. So I merged the 2 parts with TS Doctor's File Merger and then pulled it through TS Doctor which gave me some warnings:

Code Auswählen Erweitern


Opening file H:\TEST\TSDoctor bug\samples_merged.ts

OS: Windows 7 x64 Service Pack 1
TSDoctor.exe V 1.2.82 (Build 03B096)
AMD Radeon HD 6570 (DISPLAY1)
AMD Radeon HD 6570 (DISPLAY2)
Supported TS source filter found  : TS Doctor FileSource (on)
Supported splitter filter found   : Haali Media Splitter, LAV Splitter
Supported audio filter found      : LAV Audio Decoder, Microsoft DTV-DVD Audio Decoder
Supported Mpeg video filter found : LAV Video Decoder, Microsoft DTV-DVD Video Decoder
Supported H264 video filter found : LAV Video Decoder, Microsoft DTV-DVD Video Decoder
Supported video renderer found    : Video Renderer, Enhanced Video Renderer

File size: 12302720
Packets  : 65440


Scanning for TRP fill packets

Scanning for PIDs

Found 6 different PIDs
Broadcast standard: DVB

Identifying stream types

$0042: 0%   = PMT
$01FF: 92%  = H264 Video (PES_StreamID E0 = Video_Stream_0) [PCR,PTS,DTS]
$0140: 3%   = AC3 Audio (PES_StreamID BD = Private_Stream_1) [PTS][PESLength]
$0141: 3%   = AC3 Audio (PES_StreamID BD = Private_Stream_1) [PTS][PESLength]
$0120: 2%   = MPEG1 Audio (PES_StreamID C8 = Audio_Stream_8) [PTS][PESLength]
$0000: 0%   = PAT


Scanning for PATs

----------- PAT 0 -----------
PacketCount              : 1
Table ID                 : 0
Section Syntax Indicator : 1
Reserved Bit 0           : 0
Reserved Bit 1           : 1
Reserved Bit 2           : 1
Pointer Field            : 0
Section Length           : 13
Transport Stream ID      : 2
Reserved Bit 3           : 1
Reserved Bit 4           : 1
Version Number           : 0
Current Next Indicator   : 1
Section Number           : 0
Last Section Number      : 0

  Program Number : $0003
  Map PID        : $0042

CRC32                    : $E8B7E8E8 = $E8B7E8E8
-----------------------------

Found 1 different PATs
Found 1 different PMT PIDs
Scanning for PMTs

----------- PMT 0 -----------
Packet                   : $00000001
PacketCount              : 1
PID                      : $0042
Table ID                 : 2
Section Syntax Indicator : 1
PMT Size                 : 81
Pointer Field            : 0
Section Length           : 73
Program Number           : $0003
Version Number           : 0
Current Next Indicator   : 1
Section Number           : 0
Last Section Number      : 0
PCR PID                  : $01FF

Program Info Length      : 0

  Stream Type    : 27
  Description    : AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video
  Elementary PID : $01FF
  ES Info Length : 3
  ES Info        : $11, $01, $FF
  ES Info Text   : __ÿ

  Stream Type    : 3
  Description    : ISO/IEC 11172 Audio
  Elementary PID : $0120
  ES Info Length : 6
  ES Info        : $0A, $04, $64, $75, $74, $00
  ES Info Text   : __dut_

  Stream Type    : 6
  Description    : ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  Elementary PID : $0140
  ES Info Length : 16
  ES Info        : $0A, $04, $65, $6E, $32, $00, $05, $04, $41, $43, $2D, $33, $6A, $02, $80, $42
  ES Info Text   : __en2___AC-3j_€B

  Stream Type    : 6
  Description    : ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  Elementary PID : $0141
  ES Info Length : 15
  ES Info        : $0A, $04, $6E, $6C, $64, $00, $05, $04, $41, $43, $2D, $33, $6A, $01, $00
  ES Info Text   : __nld___AC-3j__

CRC32                    : $6A35C8A3 = $6A35C8A3
-----------------------------

Found 1 different PMTs

Selecting PMT with PID 0042 at position 00000001
CRC OK!

0.
  stream_type              : 27 = AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video
  elementary_pid           : 01FF
  ES_info_length           : 3

1.
  stream_type              : 3 = ISO/IEC 11172 Audio
  elementary_pid           : 0120
  ES_info_length           : 6

2.
  stream_type              : 6 = ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  elementary_pid           : 0140
  ES_info_length           : 16

3.
  stream_type              : 6 = ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  elementary_pid           : 0141
  ES_info_length           : 15

PCR PID is 01FF

Created PAT:

----------- PAT 0 -----------
PacketCount              : 0
Table ID                 : 0
Section Syntax Indicator : 1
Reserved Bit 0           : 0
Reserved Bit 1           : 1
Reserved Bit 2           : 1
Pointer Field            : 0
Section Length           : 13
Transport Stream ID      : 2
Reserved Bit 3           : 1
Reserved Bit 4           : 1
Version Number           : 0
Current Next Indicator   : 1
Section Number           : 0
Last Section Number      : 0

  Program Number : $0003
  Map PID        : $0042

CRC32                    : $E8B7E8E8 = $E8B7E8E8
-----------------------------



Created PMT:

----------- PMT 0 -----------
Packet                   : $00000001
PacketCount              : 1
PID                      : $0042
Table ID                 : 2
Section Syntax Indicator : 1
PMT Size                 : 81
Pointer Field            : 0
Section Length           : 73
Program Number           : $0003
Version Number           : 0
Current Next Indicator   : 1
Section Number           : 0
Last Section Number      : 0
PCR PID                  : $01FF

Program Info Length      : 0

  Stream Type    : 27
  Description    : AVC video stream as defined in ITU-T Rec. H.264 | ISO/IEC 14496-10 Video
  Elementary PID : $01FF
  ES Info Length : 3
  ES Info        : $11, $01, $FF
  ES Info Text   : __ÿ

  Stream Type    : 3
  Description    : ISO/IEC 11172 Audio
  Elementary PID : $0120
  ES Info Length : 6
  ES Info        : $0A, $04, $64, $75, $74, $00
  ES Info Text   : __dut_

  Stream Type    : 6
  Description    : ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  Elementary PID : $0140
  ES Info Length : 16
  ES Info        : $0A, $04, $65, $6E, $32, $00, $05, $04, $41, $43, $2D, $33, $6A, $02, $80, $42
  ES Info Text   : __en2___AC-3j_€B

  Stream Type    : 6
  Description    : ITU-T Rec. H.222.0 | ISO/IEC 13818-1 PES packets containing private data (AC3)
  Elementary PID : $0141
  ES Info Length : 15
  ES Info        : $0A, $04, $6E, $6C, $64, $00, $05, $04, $41, $43, $2D, $33, $6A, $01, $00
  ES Info Text   : __nld___AC-3j__

CRC32                    : $6A35C8A3 = $6A35C8A3
-----------------------------


First video PTS is 177649931 00:32:53.888
Last video PTS  is 178562178 00:33:04.024

First PCR  is 53256608100 00:32:52.467
Last PCR  is 53542106599 00:33:03.041
Duration of video stream is 951662 00:00:10.574
Strange PCR Jump: 00:32:52.596 -> 00:32:52.876
Strange PCR Jump: 00:32:57.848 -> 00:32:58.945
Strange PCR Jump: 00:32:59.050 -> 00:32:59.365
Video PCR to PTS difference -1421 ms
$0120: Delay to video stream = 109ms
$0140: Delay to video stream = 19ms
$0141: Delay to video stream = 32ms


The following PIDs will be imported into new stream: $01FF, 0120, 0140, 0141

Scanning for video details

-------- H264 Video ---------
profile_idc                            : 100
constrained_set0_flag                  : 0
constrained_set1_flag                  : 0
constrained_set2_flag                  : 0
constrained_set3_flag                  : 0
reserved_zero_4bits                    : 0
level_idc                              : 40
seq_parameter_set_id                   : 0
chroma_format_idc                      : 1
bit_depth_luma_minus8                  : 0
bit_depth_chroma_minus8                : 0
qpprime_y_zero_transform_bypass_flag   : 0
seq_scaling_matrix_present_flag        : 1
seq_scaling_list_present_flag_0      : 1
seq_scaling_list_present_flag_1      : 1
seq_scaling_list_present_flag_2      : 0
seq_scaling_list_present_flag_3      : 0
seq_scaling_list_present_flag_4      : 1
seq_scaling_list_present_flag_5      : 0
seq_scaling_list_present_flag_6      : 1
seq_scaling_list_present_flag_7      : 0
log2_max_frame_num_minus4              : 2
pic_order_cnt_type                     : 0
log2_max_pic_order_cnt_lsb_minus4      : 4
num_ref_frames                         : 4
gaps_in_frame_num_value_allowed_flag   : 0
pic_width_in_mbs_minus1                : 119
pic_height_in_map_units_minus1         : 33
frame_mbs_only_flag                    : 0
mb_adaptive_frame_field_flag           : 1
direct_8x8_inference_flag              : 1
frame_cropping_flag                    : 1
frame_crop_left_offset                 : 0
frame_crop_right_offset                : 0
frame_crop_top_offset                  : 0
frame_crop_bottom_offset               : 2
vui_parameters_present_flag            : 1
aspect_ratio_info_present_flag         : 1
aspect_ratio_idv                       : 1
overscan_info_present_flag             : 1
overscan_appropriate_flag              : 1
video_signal_type_present_flag         : 1
video_format                           : 5
video_full_range_flag                  : 0
colour_description_present_flag        : 1
colour_primaries                       : 1
transfer_characteristics               : 1
matrix_coefficients                    : 1
chroma_loc_info_present_flag           : 1
chroma_sample_loc_type_top_field       : 0
chroma_sample_loc_type_bottom_field    : 0
timing_info_present_flag               : 1
num_units_in_tick                      : 1
time_scale                             : 50
fixed_frame_rate_flag                  : 1
nal_hrd_parameters_present_flag        : 0
vcl_hrd_parameters_present_flag        : 0
-----------------------------

Video format: H264 1920x1088i/AR=16:9/25 fps/High@4.0
Colorimetry : ColourPrimaries=BT.709, TransferCharacteristics=BT.709, MatrixCoefficients=BT.709
First I-Frame PTS at 00:32:53.888 [00:00:00.000]
First GOP = 17 frames : I0 P16 B8 B4 B2 B6 B12 B10 B14 P32 B24 B20 B18 B22 B28 B26 B30

Scanning for audio details
Audio stream 1: MPEG1 Audio
AC3 6 channels: 18 times
Audio stream 2: AC3 5.1 48000Hz (EN2)
AC3 6 channels: 18 times
Audio stream 3: AC3 5.1 48000Hz (NLD)

No cutting


Cut in  at PCR: 00:00:00.000 (00:32:52.467)
Cut out at PCR: 00:00:10.574 (00:33:03.041)
First packet  : 00000002
Last packet   : 0000FFA0

Starting at paket 00000002 PCR: 00:00:00.000 (00:32:52.467)
TS  WARNING: For PID 01FF 00:00:05.381 TS packet 00008F7B: Packet discontinuity  last=10 , current=9
TS Warning: PCR timing for PID 01FF last PCR:00:32:57.848 current PCR:00:32:58.848 at paket 00008F7B TimeBase:178096278 PCRExtension:191
TS  WARNING: For PID 0140 00:00:07.563 TS packet 0000A896: Packet discontinuity  last=15 , current=2
TS  WARNING: For PID 0141 00:00:07.668 TS packet 0000AB4B: Packet discontinuity  last=7 , current=14
TS  WARNING: For PID 0120 00:00:07.703 TS packet 0000ABEF: Packet discontinuity  last=15 , current=2

---------  NAL Unit Type Statistic  ---------
Slices                 : 1260
    I-Slices           : 0
    P-Slices           : 126
    B-Slices           : 1134
    SP-Slices          : 0
    SI-Slices          : 0
Data Partition A       : 0
Data Partition B       : 0
Data Partition A       : 0
IDR Picture            : 66
SEI                    : 221
Sequence Parameter Set : 11
Picture Parameter Set  : 11
AUD                    : 221
End of Sequence        : 0
End of Stream          : 0
Filler                 : 166
Slices                 : 0
Seq. Param. Set Ext.   : 0

H264 filler data: 0,1% [Netto]

Cutted pakets at the beginning: 0
Cutted pakets at the end: 0
Discarded pakets (not needed): 14

PID stream sizes
$01FF: 10,7 MB
$0120: 188,9 KB
$0140: 413,3 KB
$0141: 408,1 KB

ERRORS : 0
WARNINGS : 5

Speed: 8,8 MBytes/sec
Duration: 00:00:01

I'd like to encode the file with MeGUI (which uses MKVMerge) but I get an MKVMerge warning:

Code Auswählen Erweitern

"Warning: 'H:\TEST\TSDoctor bug\samples_merged_fixed.ts' track 1: This MPEG audio track contains 536 bytes of non-MP3 data which were skipped. The audio/video synchronization may have been lost."

The encoding can't continue.

Could you fix this so TS Doctor produces a file that is completely clean and free of errors/warnings?

Logs and samples:
https://www.dropbox.com/s/r36v1nucbrunuqp/20130524%20TS%20Doctor%20warnings.7z

[Cypheros]

Use the TS Remuxer under Tools to create a MKV with synchronized audio stream (has a lipsync feature for AC3).

[Sjakko]

Thanks! After installing tsmuxer it worked.

Another thing I noticed is that after merging the 2 TS files, I have to pull it through TS Doctor twice to get most warnings out:

After 1st time 5 warnings (samples_merged_fixed.ts):

Code Auswählen Erweitern

Starting at paket 00000002 PCR: 00:00:00.000 (00:32:52.467)
TS  WARNING: For PID 01FF 00:00:05.381 TS packet 00008F7B: Packet discontinuity  last=10 , current=9
TS Warning: PCR timing for PID 01FF last PCR:00:32:57.848 current PCR:00:32:58.848 at paket 00008F7B TimeBase:178096278 PCRExtension:191
TS  WARNING: For PID 0140 00:00:07.563 TS packet 0000A896: Packet discontinuity  last=15 , current=2
TS  WARNING: For PID 0141 00:00:07.668 TS packet 0000AB4B: Packet discontinuity  last=7 , current=14
TS  WARNING: For PID 0120 00:00:07.703 TS packet 0000ABEF: Packet discontinuity  last=15 , current=2

After 2nd time only 1 warning (samples_merged_fixed_fixed.ts):

Code Auswählen Erweitern

Starting at paket 00000002 PCR: 00:00:00.000 (00:32:52.467)
TS Warning: PCR timing for PID 01FF last PCR:00:32:57.848 current PCR:00:32:58.848 at paket 00008F79 TimeBase:178096278 PCRExtension:191

Is that normal behavior?

[Cypheros]

Yes it's normal because the first time, the TS-Doctors shows the problems and fix most of them.

In the second run, the problems are already fixed and you get less errors and warnings. As an alternative, you can run "Only checking" instead of "Save new file" with the same result.

[Sjakko]

Ah right. I guess it was confusing to me that errors/warnings TS Doctor could fix and couldn't fix aren't specified separately. That way you don't know if further fixing (like remuxing) is required until you pull the fixed file through TS Doctor. Maybe it's a good idea to specify errors/warnings that TS Doctor could and couldn't fix separately? Or offer a remux if needed?

[Cypheros]

There are should be no further fixing needed. The way the TS-Doctor fixes a file will make it playable and reduces the disturbances to a minimum. Of course, some errors will remain in the file because they are not fixable (for example lost or demaged audio/video data) or would make these small errors more visible or hearable. Some artifacts in a single frame is not as bad as 2 missing seconds in a movie, if the TS-Doctor would automatically cut the problem area.

The use of a remuxer on a file with errors will probably create even more problems and in many cases audio will no longer be in sync to the video. If you use the TS-Doctor remuxer, you can circumvent this problem for AC3, EAC3 and AAC (new) because the TS-Doctor will fill audio gaps to keep audio and video in sync, what most of the other remuxers will not do.

But the whole remuxing is only needed if you want to convert the container format. I would recommend to keep the file in TS format because the format is well supported on common media players and as it's a fault tolerant format that forgives small errors.