Difficulties when errors occur with packet filter or verification only process

[TUK]

Hi,

1 / I have some difficulties when after  getting a fixed file,  there are some errors ( video or audio ). When there are let's say more than 3 errors,  when playing the video it stops for a while ( 10s or more) and continues to play. In this case  I use the TS packet filter tool but it suppresses from time to time a large sequence like ~ 10 s  although it shows a local error lasting much less than 1 s. I would expect that it suppresses some tenth of a second not more ?

2/ After having updated TS doctor (issue 134) and LAV filters ( issue 057 > 062 ), I tried to compare the number of errors of a new "fixed" file by using the "verification only" processing : I saw much more errors during the processing and no results at the end ( the software continues to run but never stops). That problem was may be existing before but I cannot be sure.

I redo several times the processing to get other fixed files always with the same original video  always  showing at the end 6 errors, but again impossible to get any result when  applying  "verification only" processing. After several tries,  It worked only once  and  this time the 6 video errors disapeared but not the 2 or three audio ?
How these discrepencies can be explained ?

Thanks for clarifications.

[Cypheros]

Hi, sorry but I have no idea what the problem is. Could you post the logs to such a problem file?

[TUK]

Hello,
- Thanks for your reply

- Problem of 10s video interruption after application of filter packet : I need to find proper files and I need some time.

- Reproducibility of Check function :
     see attached documents screenshots and log files ( sent in 2 posts )

I you need anu further materials please tell me

[TUK]

Part 2 of attached documents

[TUK]

Hello,

On my  post I indicated 2 types of difficulties :

        - "Check only" function : no answer although logs posted ....

        - 10s interruptions on « fixed » video

Having implemented the last issue of TS doctor, this last problem do not appear anymore  the same way. However I still would like to have your advice. One of my last  records, presents a large number of video  errors concentrated on 2 spots (2 red lines during the processing ), the basic results of the processings are as follow :

    a/original video : on screen 2 bad arias ( much less than a second), Check function = 238 errors
    b/After cut & fixed vidéo : on screen 2 bad areas again,  236 errors  ,
    c/After cut & fixed & filtered :  on screen 2 bad areas always but only 7 audio errors 

Question 1 : The main result is that I get a video without advertisement but no ( visual)change with respect to the bad areas. Is there any other method to improve the situation than apply in final the packet filter function ?

Question 2 : in the case of such few and short bad areas a solution (?) could be to cut the few bad frames detected with « Check only ». It would imply a cutter able to operate with the packet numbers( start and end of the bad areas) ?
I succeeded to find the bad packets using the TS packet editor. I found 3 different packets full of « FF » data which apparently correspond with the bad areas but noway to suppress them !

[Cypheros]

Thanks for the logs. This is a difficult problem and I have to analyse the logs to see the problem.

Under normal circumstances, no packet filtering should be needed. TS-Doctor is analysing the stream and only copy valid packets. I have to check why the packet filter removes aditionally (bad) packets.

I guess it's because the packet filter is not as smart as the main routine. Sometimes packets are marked as bad (invalid or encrypted) but maybe are not. The packet filter will remove all bad marked packets without thinking about it.

The main TS-Doctor routine will only remove packets if they are bad for sure. There are some cases where TS-Doctor can't be sure, if a packet is demaged or just marked falsely as demaged and then will keep this packets. Only during playback you will see if there is demage or not.

[TUK]

Hi,
- Thanks for your reply

- Only on video presenting some local problems, I used the packet filter tool for the following reasons :

       - the  file cannot be played with VLC ( but is usualy OK with Windows media )
       - It's Playing présents  interruptions ( 10s for example)
       - Taking as reference the number of errors ( log of fixed file or Check test ), I noticed that the number of errors is divided by a significant value and the playing of filtered video  is often ( but not always )  easier.

- Concerning my suggestion to cut these small bad areas : possible ?

Regards

[TUK]

Hi,

I sent you logs but to day no answer, the analysis seems not easy.
I did on my side some other tests which could be of some help, I hope...Theses tests are very simple :

1/ I test an original  record ( 2h 52') presenting some defaults with the « Check » function
  The results is 46 errors ( 34 video ). The «  fehler verteilung » or distribution of errors if the translation is correct, shows one error at the beginning and  numerous errors at the end.
The record can be played with VLC.

             2/ I use now the main routine and I cut 2 areas. The fixed file ( 2h 37' ) presents now an incredible number of errors ( 712795!). The  distribution of errors  shows now clearly new errors not shown previously in the middle of the record !
             That new record cannot be played with VLC only with Windows media.

              3/ Same test than previously but I do only one cut at the beginning ( 2h50'). Result only 18 errors  and a distribution looks very much like the original (1).
               However the new record cannot be played with VLC only with Windows media.

I am convince now that these problems occur not on a systematic way but they occur ( it is not the first time and that's why I used the packet filtertool  to try to improve the situation).
The question is how TSdoctor can generate errors or is it due to my PC ?

  Best regards     

[Cypheros]

Very strange problem. I looks like the are packets missing. It could be a datatransfer problem.

What is drive F:?
Is it a local or a network drive?

You are using TS-Doctor in idle mode, did you try to use it with normal priority?

[TUK]

HI,
   Thanks for your reply

     - F is an external WD MYbook drive 2To connected to my PC via  USB2 port.

    - No Idle use idle mode not knowing the difference between all possibilities, any reason to change ?

[Cypheros]

Maybe it's a priority problem where the transfer to the WD is loosing data. Try to check the CRC you can find in the log with the CRC you can get from the final file using TS-Doctor/Tools/File CRC. If the CRC is different, there was a data transfer error.

Data transfer to USB or network drives are not save. There is no indicator for data loss during the transfer.

[TUK]

Hi again,

- Concerning the priority, please tell me if I have any reason to change from Idle to any other mode ?

- In the 3 cases analysed previously I join 2 of  the corresponding logs because I do not understand what to do with such a result , as an example  :
CRC32     : $883BA196 = $883BA196 ?
The tool used with one of these file CRC gives me one result  " CRC = AECD6ACF" to be compared to what figure ?

Any tuto on this subject( which seems a good indicator) ?

[Cypheros]

Priority should have no influence but it was just a idea to check that.

If you create a new file you can find at the end of the log the file list with CRCs:

Code Auswählen Erweitern

File sizes:
            F:\A_Policiers_3\Dysfonctionnements\Nouveau dossier\dossier à envoyer\1_original_fixed_4.ts 4,11 Go [CRC=B396A076]

Now you can use the File CRC tool on the file "F:\A_Policiers_3\Dysfonctionnements\Nouveau dossier\dossier à envoyer\1_original_fixed_4.ts" to check if you get the same CRC (B396A076) as in the log.
If the CRCs are identical, the file transfer was OK and your have on your drive exactly what TS-Doctor has written to the drive. If they are differnt, the data transfer to the drive failed.

[TUK]

HI,
- Thanks very much, I will now have a look on this CRC parameter . I let the priority "normal" instead of "idle" even if I did not see any influence for the moment.

- Data transfer :I get my original videos either from an external HDD  connected  to a player  or via my LiveBox, in that case the video records are tranferred to my PC via an ethernet câble.

Expecting that these data transfer do not present too many risks ( but I do not see how to improve that situation),  I intend now, to transfer the data to an internal hard disk of my PC to "fix" a file with TS doctor. I hope that this direct way will solve a good part of the problems observed untill now.

Please confirm that, on a theoritical point of view,  I shall proceed this way (I try to check the saying " when there is a will there is a way"  )...

[Cypheros]

Yes, both ways should work but maybe there is a hidden problem, we haven't found till now.

If I look into the logs, I can see that after fixing there are errors in the file like sync byte and discontinuety errors. This kind of errors should not happen because the writing function of TS-Doctor writes TS pakets with correct formating (sync byte) and discontinuety_counter in any case.

If there is such an error in the resulting file, the file is corrupt and the data TS-Doctor has written was not transfered to the disk correctly.