Hi,
1 / I have some difficulties when after getting a fixed file, there are some errors ( video or audio ). When there are let's say more than 3 errors, when playing the video it stops for a while ( 10s or more) and continues to play. In this case I use the TS packet filter tool but it suppresses from time to time a large sequence like ~ 10 s although it shows a local error lasting much less than 1 s. I would expect that it suppresses some tenth of a second not more ?
2/ After having updated TS doctor (issue 134) and LAV filters ( issue 057 > 062 ), I tried to compare the number of errors of a new "fixed" file by using the "verification only" processing : I saw much more errors during the processing and no results at the end ( the software continues to run but never stops). That problem was may be existing before but I cannot be sure.
I redo several times the processing to get other fixed files always with the same original video always showing at the end 6 errors, but again impossible to get any result when applying "verification only" processing. After several tries, It worked only once and this time the 6 video errors disapeared but not the 2 or three audio ?
How these discrepencies can be explained ?
Thanks for clarifications.
Hi, sorry but I have no idea what the problem is. Could you post the logs to such a problem file?
Hello,
- Thanks for your reply
- Problem of 10s video interruption after application of filter packet : I need to find proper files and I need some time.
- Reproducibility of Check function :
see attached documents screenshots and log files ( sent in 2 posts )
I you need anu further materials please tell me
Part 2 of attached documents
Hello,
On my post I indicated 2 types of difficulties :
- "Check only" function : no answer although logs posted ....
- 10s interruptions on « fixed » video
Having implemented the last issue of TS doctor, this last problem do not appear anymore the same way. However I still would like to have your advice. One of my last records, presents a large number of video errors concentrated on 2 spots (2 red lines during the processing ), the basic results of the processings are as follow :
a/original video : on screen 2 bad arias ( much less than a second), Check function = 238 errors
b/After cut & fixed vidéo : on screen 2 bad areas again, 236 errors ,
c/After cut & fixed & filtered : on screen 2 bad areas always but only 7 audio errors
Question 1 : The main result is that I get a video without advertisement but no ( visual)change with respect to the bad areas. Is there any other method to improve the situation than apply in final the packet filter function ?
Question 2 : in the case of such few and short bad areas a solution (?) could be to cut the few bad frames detected with « Check only ». It would imply a cutter able to operate with the packet numbers( start and end of the bad areas) ?
I succeeded to find the bad packets using the TS packet editor. I found 3 different packets full of « FF » data which apparently correspond with the bad areas but noway to suppress them !
Thanks for the logs. This is a difficult problem and I have to analyse the logs to see the problem.
Under normal circumstances, no packet filtering should be needed. TS-Doctor is analysing the stream and only copy valid packets. I have to check why the packet filter removes aditionally (bad) packets.
I guess it's because the packet filter is not as smart as the main routine. Sometimes packets are marked as bad (invalid or encrypted) but maybe are not. The packet filter will remove all bad marked packets without thinking about it.
The main TS-Doctor routine will only remove packets if they are bad for sure. There are some cases where TS-Doctor can't be sure, if a packet is demaged or just marked falsely as demaged and then will keep this packets. Only during playback you will see if there is demage or not.
Hi,
- Thanks for your reply
- Only on video presenting some local problems, I used the packet filter tool for the following reasons :
- the file cannot be played with VLC ( but is usualy OK with Windows media )
- It's Playing présents interruptions ( 10s for example)
- Taking as reference the number of errors ( log of fixed file or Check test ), I noticed that the number of errors is divided by a significant value and the playing of filtered video is often ( but not always ) easier.
- Concerning my suggestion to cut these small bad areas : possible ?
Regards
Hi,
I sent you logs but to day no answer, the analysis seems not easy.
I did on my side some other tests which could be of some help, I hope...Theses tests are very simple :
1/ I test an original record ( 2h 52') presenting some defaults with the « Check » function
The results is 46 errors ( 34 video ). The « fehler verteilung » or distribution of errors if the translation is correct, shows one error at the beginning and numerous errors at the end.
The record can be played with VLC.
2/ I use now the main routine and I cut 2 areas. The fixed file ( 2h 37' ) presents now an incredible number of errors ( 712795!). The distribution of errors shows now clearly new errors not shown previously in the middle of the record !
That new record cannot be played with VLC only with Windows media.
3/ Same test than previously but I do only one cut at the beginning ( 2h50'). Result only 18 errors and a distribution looks very much like the original (1).
However the new record cannot be played with VLC only with Windows media.
I am convince now that these problems occur not on a systematic way but they occur ( it is not the first time and that's why I used the packet filtertool to try to improve the situation).
The question is how TSdoctor can generate errors or is it due to my PC ?
Best regards
Very strange problem. I looks like the are packets missing. It could be a datatransfer problem.
What is drive F:?
Is it a local or a network drive?
You are using TS-Doctor in idle mode, did you try to use it with normal priority?
HI,
Thanks for your reply
- F is an external WD MYbook drive 2To connected to my PC via USB2 port.
- No Idle use idle mode not knowing the difference between all possibilities, any reason to change ?
Maybe it's a priority problem where the transfer to the WD is loosing data. Try to check the CRC you can find in the log with the CRC you can get from the final file using TS-Doctor/Tools/File CRC. If the CRC is different, there was a data transfer error.
Data transfer to USB or network drives are not save. There is no indicator for data loss during the transfer.
Hi again,
- Concerning the priority, please tell me if I have any reason to change from Idle to any other mode ?
- In the 3 cases analysed previously I join 2 of the corresponding logs because I do not understand what to do with such a result , as an example :
CRC32 : $883BA196 = $883BA196 ?
The tool used with one of these file CRC gives me one result " CRC = AECD6ACF" to be compared to what figure ?
Any tuto on this subject( which seems a good indicator) ?
Priority should have no influence but it was just a idea to check that.
If you create a new file you can find at the end of the log the file list with CRCs:
File sizes:
F:\A_Policiers_3\Dysfonctionnements\Nouveau dossier\dossier à envoyer\1_original_fixed_4.ts 4,11 Go [CRC=B396A076]
Now you can use the File CRC tool on the file "F:\A_Policiers_3\Dysfonctionnements\Nouveau dossier\dossier à envoyer\1_original_fixed_4.ts" to check if you get the same CRC (B396A076) as in the log.
If the CRCs are identical, the file transfer was OK and your have on your drive exactly what TS-Doctor has written to the drive. If they are differnt, the data transfer to the drive failed.
HI,
- Thanks very much, I will now have a look on this CRC parameter . I let the priority "normal" instead of "idle" even if I did not see any influence for the moment.
- Data transfer :I get my original videos either from an external HDD connected to a player or via my LiveBox, in that case the video records are tranferred to my PC via an ethernet câble.
Expecting that these data transfer do not present too many risks ( but I do not see how to improve that situation), I intend now, to transfer the data to an internal hard disk of my PC to "fix" a file with TS doctor. I hope that this direct way will solve a good part of the problems observed untill now.
Please confirm that, on a theoritical point of view, I shall proceed this way (I try to check the saying " when there is a will there is a way" )...
Yes, both ways should work but maybe there is a hidden problem, we haven't found till now.
If I look into the logs, I can see that after fixing there are errors in the file like sync byte and discontinuety errors. This kind of errors should not happen because the writing function of TS-Doctor writes TS pakets with correct formating (sync byte) and discontinuety_counter in any case.
If there is such an error in the resulting file, the file is corrupt and the data TS-Doctor has written was not transfered to the disk correctly.
Hi,
I did not join, last time, the log of the file presenting ~75000 errors ( see jpeg snapshot) because it was a log of 35 Mo !
I suppressed in this file most of the lines such "TS ERROR : For PID 00DC, invalid packet 00E38772! Error: sync_byte_error to get a reasonable size.
I hope that's will help you ?
This recording is very demaged. As you can see, the paket numbers are consecutive. So there are bigger areas of demaged data.
Try to copy the original recording to your local hard disk (C:\) and if open this local copy and create a new file, make sure you write it back to the same local drive (C:\). Make a check of the resulting file and tell me if this file is demaged, too.
Hi,
- Always on the same original record, I did as you suggested, further tests as follow on both C drive and in addition with an internal HDD. I transferred then my original record ( stored on my external WD HDD ) on these 2 drives. I made then a check of the transferred file before applying the routine and after.
.
- Test results in terms of warnings/errors are shown in the first image ( test_summary. It shows :
* a large improvment when using C or Internal drives instead of external HDD.
* No difference between C or the internal HDD
* something I do not understand : The results after processing ( 17 warnings/11 errors) becomes 0 warning and only 3 errors after checking the "fixed" file, Why ?
It seems that this check do more than checking : I noticed in log after check test of the original record the following :
Working in M2TS mode with 192 byte packets!
Pakets will be converted from 192 byte packets to 188 byte packets!
File size: 4626614592
Packets : 24096951 etc....
and also at the end of the log : "Discarded packets (not needed): 1627269 , Discarded invalid packets : 19 etc. "
- However the major point is that it really seems to me that the use of an external drive with an USB interface is the origin of my problems. As I said previously I proceed differently and do not use external drives to process the data. When I will have accumulated enough results concerning the future records I will let you know.
I will appreciate your comments and if possible an answer to the surprising "check tests results"
Zitat* something I do not understand : The results after processing ( 17 warnings/11 errors) becomes 0 warning and only 3 errors after checking the "fixed" file, Why ?
Most of the problem will be fixed by TS-Doctor but some not. In your case I guess the AC3 CRC errors are not fixed. Replacing the demaged AC3 frame with silence or override of the CRC can make the problemes worse and more noticeable.
Hi,
I do not quite understand your last comment :
- The situation before processing was 60 warnings, 40 errors ( 34 video + 3 audio Mpeg1)
- After processing the fixed file shows 17 warnings/11 errors (6 video + 2 audio Mpeg1) which is a large improvment, everything OK at this point.
- If I apply now to the fixed file the check test, it shows at the end 0 warning and only 2 audio errors.
I would have expected to find the same numbers of errors after check than before (I suppose that no improvment can occur after a check test) ?
OK, I see what you mean. After processing TS-Doctor shows the errors found during processing the file but no counter for the fixed or remaining errors. Yes that's true.
The errors are fixed during the first processing of course, not the check test.
Hi,
- OK, the informations given after processing do not give the real situation. If one needs to know it, it is mandatory to perform Check test .
- In the table already communicated (test_summary.jpeg ), there were 3 columns : check test before and after processig + the result of the processing. This intermediate column do not then give a reliable information to measure the performance of the routine. It is then necessary to perform 2 Check tests to have it ( before & after).
- I made several records processings since a few days on an internal HDD and ( I am crossing my fingers) no more errors as in the past. I do not know if it is said in a manual but this should indicated in red " never use an external HDD to save and process the data" !
Thanks anyway for your efforts and explanations