I recently upgraded the version to the latest full build 1.2.22 and now have a problem with running within a scheduled task.
I have a batch script I run to convert all my tv recording every night, and it initially runs TSdoctor to clean up the recording before converting to an mkv. TSdoctor seems to run ok, it creates the fixed file but it just won't close again automatically. I reverted back to 1.1.35 and all is fine again. I only run with the autofix parameter e.g
"C:\Program Files (x86)\Cypheros\TSDoctor\TSDoctor.exe" "\\mediaserver\TV Recordings\Stand Up for the Week\Stand Up for the Week - S00E00 - 2012-05-18_23-10.ts" AUTOFIX "\\mediaserver\TV Recordings\Stand Up for the Week\Stand Up for the Week - S00E00 - 2012-05-18_23-10_fixed.ts"
Runnign interactively from a command prompt it runs and closes OK.
Any thoughts as its frustrating not being able to use the latest build. I tried running with the LOG parameter but it just creates an empty log file so no indications of what could be wrong.
Hi, please check the filename. Maybe there is a typo in the source filename. Checked with network files (UNK) and found no problem with the commandline processing. Application is closing but the application will not close if the source file is not found or not accessable and no message or hint is generated.
Next version will raise a "File not found" error to prevent such problems.
The filenames are fine. Its creating the fixed version OK and completes as far as I can see, it just then doesn't seem to close. if I run manually from the command line the same command completes OK. It just seems to stick after processing.
Try to run without the "log" parameter. There seems to be a problem with the writing of the log that could stop the closing of the application. Without the "log" parameter it works for me.
"Log" will be fixed in the next version.
Tried that. Also made sure I had unticked the log option within the menus. The same thing happens, it completes the fix and then just doesn't close.
I should say this is running on a windows home server 2011 in case this makes a difference.
It would help to know at which version the problem appeared. Could you check the older version?
http://www.cypheros.de/tsdoctor_alte_versionen_e.html (http://www.cypheros.de/tsdoctor_alte_versionen_e.html)
1.1.14 - OK
1.1.46 - OK
1.2.5 - could not download from that page
1.2.16 - Does it on this version
I also checked it wasn't to do with h.264 recordings and it does the same for standard MPEG too.
Sorry, 1.2.5 should work now.
Checked and it doesn't close on 1.2.5 either.
Thanks for the tests. I'll check the change logs and try to find a solution for the problem.
I'm having the same problem, if I run the batch file interactivly TSDoctor closes as supposed, but not when it is started as an automated process started by the service that does my recordings. Any progress on this?
Thanks for the additional hint. It seems to be connected with the remote start by another program.
When can we expect a fix for this problem?
Was not able to reproduce the problem but I did some changes to the closing procedure that will make the closing/termination of the program more reliable (hope so).
Next beta 1.2.27 is comming soon.
Another thing I noticed, when I run TSDoctor in batch mode with AUTOFIX and specify an output filename, the fixed file does not get the name I specify, instead it gets the original name with _fixed added to the name.
This will happen only if the output filename/folder is invalid. Maybe the output folder does not exist.
Just tested .27 beta, but it does not end when run as a batch. I think it has to do with TSDoctor displaying a summary message with errors and warnings at the end. Is it nok possible to skip this message when run as a batch or with commandline options?
Zitat von: Cypheros am Juni 04, 2012, 15:12:29
This will happen only if the output filename/folder is invalid. Maybe the output folder does not exist.
The output directory was the same as the input, more spesific "%dpn1.fix"
Any ideas???
Changed some procedures to prevent this from happening but I was not able to reproduce the problem. Please wait for the next Beta 1.2.28 and tell me if it's working as expected.
Zitat von: Cypheros am Juni 07, 2012, 09:30:48
Changed some procedures to prevent this from happening but I was not able to reproduce the problem. Please wait for the next Beta 1.2.28 and tell me if it's working as expected.
Did you actually have a corrupted .ts file to test with? I can supply one if you need.
Corrupted? Is there something special with the files producing the problem? I was not aware it.
Yes indeed, it would help to have such a file. I could give you access to a FTP server if you want.
Zitat von: Cypheros am Juni 07, 2012, 23:29:35
Corrupted? Is there something special with the files producing the problem? I was not aware it.
Yes indeed, it would help to have such a file. I could give you access to a FTP server if you want.
Just my thought that a clean file won't produce a message popup at the end. Have you removed all popups in batch mode?
A 200Mb chunk can be found at ftp://orion.servebeer.com, login as anonymous and use your email as password, go to the folder TSDoctor and download "The Blue Planet.ts"
Sorry, no TSDoctor folder found.
Zitat von: Cypheros am Juni 08, 2012, 16:26:14
Sorry, no TSDoctor folder found.
Sorry, put in the wrong folder. It is there now :-[
Were you able to reproduce the problem?
I don't know if it's related to your problem but found a strange behaviour of the application, processing this file.
Well, the TSDorcor is called by a batch file which gets the filename as its first parameter: "TSDoctor %1 AUTOFIX %dpn1.fix" is the commandline, and the batch is again called from a system service. You could try and reproduce that.
Zitat von: Cypheros am Juni 09, 2012, 14:44:34
I don't know if it's related to your problem but found a strange behaviour of the application, processing this file.
That reminds me of another thing I'v been meaning to mention, when processing recordings from BBC Knowledge TSDoctor shows only one teletext subtitle stream, even thou there are several.
There are 3 subtitle pages but they are all from one single stream, the teletext stream.
You can see the detected teletext subtitle pages in the log or by pressing the "change language descriptor" button of the teletext stream.
But the languages were not detected correctly and the teletext subtitle listbox is not scrollable. I will fix that.
Zitat von: Cypheros am Juni 09, 2012, 19:57:11
There are 3 subtitle pages but they are all from one single stream, the teletext stream.
You can see the detected teletext subtitle pages in the log or by pressing the "change language descriptor" button of the teletext stream.
But the languages were not detected correctly and the teletext subtitle listbox is not scrollable. I will fix that.
Yes ofcourse they are from the same stream, my mistake. Nice to get that fixed thou. Would it be possible to have a commandline parameter to extract .srt subtitles to a file with the same name as the input but with .srt extension? Possible EXTRACTSRT=<pageno>
Hi you can activate the SRT export unter Settings/Teletext and select the file name extension.
Zitat von: Cypheros am Juni 09, 2012, 21:56:11
Hi you can activate the SRT export unter Settings/Teletext and select the file name extension.
But that would export all pages? With pagenumber included in the filename?
Yes, it will export all page found in the recording. You can change the extension to include the page number oder you can filter the pages, so only the page numbers you want, will be exported.
Zitat von: Cypheros am Juni 10, 2012, 00:23:37
Yes, it will export all page found in the recording. You can change the extension to include the page number oder you can filter the pages, so only the page numbers you want, will be exported.
That is most unfortunate since the file will not be recognized by NPVR that I'm using. NPVR relies on the .srt file having the exact name as the recording. (www.nextpvr.com) There is a built-in teletext subtitle handling in NPVR but is does not work well since it relies on Microsoft's demultiplexer.
If you have a recording called "My fantastic Movie_fixed.ts" and you have a teletext subtitle in English on page 888, the srt will have the filename "My fantastic Movie_fixed.eng.srt". This is the default name convention and you will have working subtitles on MediaPlayerClassic, WDTV Live and Popcorn Hour A100/200 and much more.
Zitat von: Cypheros am Juni 10, 2012, 10:34:21
If you have a recording called "My fantastic Movie_fixed.ts" and you have a teletext subtitle in English on page 888, the srt will have the filename "My fantastic Movie_fixed.eng.srt". This is the default name convention and you will have working subtitles on MediaPlayerClassic, WDTV Live and Popcorn Hour A100/200 and much more.
Welll, I don't use any of them and as I said npvr will not be able to use the file without renaming first.
And how is NextPVR handling multiple subtitels?
Zitat von: Cypheros am Juni 10, 2012, 11:06:01
And how is NextPVR handling multiple subtitels?
It does not from a file, from the included subtitles there is a menu to choose from, but as I mentioned it does not work so well since it relies on the Microsoft demux. I have previously used ProjectX to extract subtitles but it is very clumsy and it also leaves the page number as a part of the filename, like File[886].srt, and this is very inconvinient.
Another thing I should mention is that NPVR records timing info along with the recordings, either as an alternate strem (in NTFS filesystem) or as a separate file, which will have the name <Recording-name>.Timing.Info. If this is present TSDoctor could use this to correct errors in the .ts file. I can provide you with a sample if you'd like?
Yes, if you provide a sample, I would look into it. Maybe I find usable data to process the file.
I'v prepared a 10 min recording and placed it in the TSDoctor folder on my ftp site. The files start with BBCKnowledge... I'll see to that you have the full bandwith to download them.
Thanks. Downloading...
Did you find any useful information in the files? Any clue when the next beta will be out?
Sorry, the timing.info is only a table of PTS values and positions. The TS-Doctor doen't need them to work because the TS-Doctor already extracts and fixes this values.
The .xml files it's just a short info about the recording (channel, time, etc.) but maybe I can use this informations to create an additional info file and combine this data with the findings in the stream.
Next beta release should be ready at the end of this week.
Zitat von: Cypheros am Juni 12, 2012, 17:07:19
Sorry, the timing.info is only a table of PTS values and positions. The TS-Doctor doen't need them to work because the TS-Doctor already extracts and fixes this values.
The .xml files it's just a short info about the recording (channel, time, etc.) but maybe I can use this informations to create an additional info file and combine this data with the findings in the stream.
Next beta release should be ready at the end of this week.
Yes the timing.info is intended for missing/inaccurate PTS values. Looking forward to test the next pre :)
BTW: Did you manage to recreate the problem with TSDoctor not quitting when run in batch mode?
No, not with the current version 1.2.29 but I could not test it with a service as host, only in a userspace environment as batch.
Zitat von: Cypheros am Juni 13, 2012, 12:43:20
No, not with the current version 1.2.29 but I could not test it with a service as host, only in a userspace environment as batch.
That would probably be a bit different, as services run as local system account, and if the problem somehow is connected to permissions you would not see it running as an ordinary user.
Zitat von: Reddwarf am Juni 13, 2012, 12:50:32
Zitat von: Cypheros am Juni 13, 2012, 12:43:20
No, not with the current version 1.2.29 but I could not test it with a service as host, only in a userspace environment as batch.
That would probably be a bit different, as services run as local system account, and if the problem somehow is connected to permissions you would not see it running as an ordinary user.
I am suffering from the same problem, as reported in the german section of the forum.
My setup involves using ForTheRecord (http://www.4therecord.eu) as a recording service on a win7_64-HomePremium install.
These services are set-up to run under real-user credentials. The same user TS-Doctor has been installed with.
Zitat von: Hominidae am Juni 13, 2012, 22:02:07
Zitat von: Reddwarf am Juni 13, 2012, 12:50:32
Zitat von: Cypheros am Juni 13, 2012, 12:43:20
No, not with the current version 1.2.29 but I could not test it with a service as host, only in a userspace environment as batch.
That would probably be a bit different, as services run as local system account, and if the problem somehow is connected to permissions you would not see it running as an ordinary user.
I am suffering from the same problem, as reported in the german section of the forum.
My setup involves using ForTheRecord (http://www.4therecord.eu) as a recording service on a win7_64-HomePremium install.
These services are set-up to run under real-user credentials. The same user TS-Doctor has been installed with.
I just tried running the service as my regular user, same thing happened.
Are any logs created?
Came to think of one thing and did some more testing. .29 actually works but the service that calls the batch that again calls TSDoctor MUST be allowed to interact with the desktop!
Yes thats right. The TS-Doctor is a GUI application and needs a desktop to create it's windows.
I guess you changed the access rights?
Zitat von: Cypheros am Juni 16, 2012, 00:28:36
Yes thats right. The TS-Doctor is a GUI application and needs a desktop to create it's windows.
I guess you changed the access rights?
At the moment the service is running as my own user, have not tried to let it run as local system yet, the system has been too busy doing recordings to do any more testing. I'll let you know when I have done more testing.
The problem with TSDoctor not terminating has definitly something to do with permissions, if I run the service as locas system account it won't terminate.
Would it be possible to have a setting to set TSDoctor's priority class? As of now it runs with NORMAL priority and that chokes my computer and prevents playing back other recordings while TSDoctor is processing.
Yes, but only with the GUI. In the menu you will find under Settings the entry "Priority" to change the priority class.
Just to update - I've loaded the latest version and now it now closes properly in a scheduled task. Thanks for resolving this.
Zitat von: Cypheros am Juni 16, 2012, 22:45:49
Yes, but only with the GUI. In the menu you will find under Settings the entry "Priotrity" to change the priority class.
Comskip has a setting called PlayNice, it defines the number of milliseconds that it should sleep between each frame it processes to easen the strain on the cpu. A similar setting in TSDoctor (for batch processing) would be most welcome :)
You want to have a wait time after processing or during processing?
If you want to slow the TS-Doctor down, you can use the GUI to change the priority class of the TS-Doctor.
Zitat von: Cypheros am Juni 18, 2012, 11:06:14
You want to have a wait time after processing or during processing?
If you want to low the TS-Doctor down, you can use the GUI to change the priority class of the TS-Doctor.
I'v tried that, but it still chokes the computer with heavy disk access, so a 10-30 mS sleep between each frame would help greatly, as it does in Comskip.
Zitat von: Reddwarf am Juni 15, 2012, 23:20:31
Came to think of one thing and did some more testing. .29 actually works but the service that calls the batch that again calls TSDoctor MUST be allowed to interact with the desktop!
....sorry for being a windoze noop...how do you apply that kind of setting to a service?
Are you running the service in session 1, while/where the user is logged in?
Zitat von: Hominidae am Juni 19, 2012, 01:10:57
Zitat von: Reddwarf am Juni 15, 2012, 23:20:31
Came to think of one thing and did some more testing. .29 actually works but the service that calls the batch that again calls TSDoctor MUST be allowed to interact with the desktop!
....sorry for being a windoze noop...how do you apply that kind of setting to a service?
Are you running the service in session 1, while/where the user is logged in?
You'll have to run Services.msc, right click on the actual service and choose Properties, then select the "Log On" tab. If you click the "This account" button you can specify which service the user runs as.
Running Services as ordenary Users usually ends in a total mess (as long, as the service is not designed this way, and since TS Doc is more an interactive desktop application (even 32 bit only), this should break things very likely).
This is because shells like the explorer usually already aquire certain system rights and inherit them to the launched application. Service manager does not do this, its up to the service himself to request that right to become active.
Unix users usually look at the file permission, maybe even a bit deeper into extended permissions and know what the application can do with these permissions. In Windoze its different, a user must have a certain right (for instance "Backup priviledge"), but this alone does not do or grant anything. The launched program also must request to use this right ( like "during my runtime I want to open a file or more with backup rights" (again unix-guys, this does not mean that it is a backup program, just that it wants to read ANY and EVERY file without looking at file permissions at all)) at startup.
Desktop programs usually dont request all rights needed, because some are already inherited as mentioned above.
So, to be on the safe side, ask the dear programmer here to ensure that EVERY needed right is reclaimed at startup time of TS Doc (and dont blame him now, if it is not (yet). remember, its unlikely and normaly not needed).
But its a hard and tedious job, because you dont see any differences instantly, and some rights are really.. hmm, how to say?, "STRANGE" to determine (for instance "Bypass traverse checking") because usually ANYBODY has them, but a service run as a user has not.
I would recommend to run your service with the account of the local system and allow "interaction with the desktop" instead. This usually gives less troubles because services are started differnently this way (not completly like desktop, but at least with "sane" rights inherited)
I think you have misunderstood a bit. It is running as a privileged user (and the user who owns the files in question) that works, WITH "Interact with desktop" allowed. Running as the local system does not work.
It is still a bit strange, the file is created in any case, but TSDoctor just will not end if run as the local system account.
Zitat von: Reddwarf am Juni 19, 2012, 19:11:05
I think you have misunderstood a bit. It is running as a privileged user (and the user who owns the files in question) that works, WITH "Interact with desktop" allowed. Running as the local system does not work.
It is still a bit strange, the file is created in any case, but TSDoctor just will not end if run as the local system account.
Nope, I've understood perfectly, you just did not understand me :-)
There is NO WAY for you to "GIVE ALL NEEDED PRIVILEDGES" to a program or a user. This is a mandantory needed thing, but not a completly sufficient one.
Windooze works a bit different, the program also needs to actively request a priv (the request is then granted if the user owns that particular priv).
So if you grant a priv to a user, it does NOTHING unless the program also asks for it.
And Services have to ask for MORE PRIVS than a standard desktop application, so it is unlikely, that TS Doc already asks for all the needed things.
One strange effect of this can be that the app "hangs" somewhere because a needed priv is not asked for but the program tries a command that needs tis priv.
Either the request is denied by windows (error return), or, in some cases, the app is halted and put into endless sleep. this can happen easily if the app tries to shutdown and one of the cleanup tasks fails due to the lack of a priv. In this case the mutex is never released and the app looks to "hang".
(there is nothing wrong with the code, just a priv has "forgotten" to be reclaimed at startup)
I understood you perfectly, but I still think we are talking across. It seems that what you last described is what is happening when TSDoctor does not terminate, but it does NOT happen when the recording service (which calls the batch that runs TSDoctor) is run as a privileged user that owns the files in question.
Zitat von: Reddwarf am Juni 19, 2012, 21:14:35
I understood you perfectly, but I still think we are talking across. It seems that what you last described is what is happening when TSDoctor does not terminate, but it does NOT happen when the recording service (which calls the batch that runs TSDoctor) is run as a privileged user that owns the files in question.
what i last described was just a guess what MIGHT happen due to a missing priv. But it will be hard to proove for the programmer because there is almost no way to debug a service in its native runtime enviroment (It might work with remote debbugging, but I never tried it).
Your recording service and the batch afterwards make things even more complicated. Both of them surely ask for SOME privs (those that are needed by them) which are granted and activated and INHERITED.
Therefor TSDoc gets a completely different runtime environment as started by a user or run as a native service.
So its almost impossible to guess for the author, WHAT is still missing.
To be on the safe side, he would have request ALL that are needed for TS_doc (but even this is quite a challenge, surely he does not know which one, this takes a very haaaard and deeep search into the programmer's manuals).
Zitat von: Reddwarf am Juni 18, 2012, 11:30:00
Zitat von: Cypheros am Juni 18, 2012, 11:06:14
You want to have a wait time after processing or during processing?
If you want to low the TS-Doctor down, you can use the GUI to change the priority class of the TS-Doctor.
I'v tried that, but it still chokes the computer with heavy disk access, so a 10-30 mS sleep between each frame would help greatly, as it does in Comskip.
OK, set application priority to IDLE and the disk I/O performance of the TS-Doctor will be reduced too. New Beta 1.2.31 is online.
Thanks, setting IDLE priority seems to have helped. In some cases I will have two or more instances of TSDoctor running at the same time, would that create a problem?
Yes, because some temporary files could use the same filenames (logs, batch, file merging,etc.).
Sorry, never thought about a multi instance scenario.
Also, many programs running at IDLE priority would obviously block each other. If a programm is doing something, its obviously NOT IDLE anymore.
Of course, it will work somehow and even finish someday, but its not predictable at all.
(in general, IDLE should be used with care, and should be limited to programs that dont use many ressources (like screensavers or so). TSDoc is surely NOT the best candidate for this because it is a real disk hog).
Also, keep in mind that IDLE is the worst thing you can do to a VM! The virtual manager needs the IDLE state to switch back to the host system or to other vms.
Zitat von: Mam am Juni 25, 2012, 08:29:06
Also, many programs running at IDLE priority would obviously block each other. If a programm is doing something, its obviously NOT IDLE anymore.
Of course, it will work somehow and even finish someday, but its not predictable at all.
(in general, IDLE should be used with care, and should be limited to programs that dont use many ressources (like screensavers or so). TSDoc is surely NOT the best candidate for this because it is a real disk hog).
Also, keep in mind that IDLE is the worst thing you can do to a VM! The virtual manager needs the IDLE state to switch back to the host system or to other vms.
In spite of that it is working fine now :)
I'm sorry to say that the latest beta does not terminate the way I run it, but 1.2.29 does work ok.
Bad news :(
Zitat von: Reddwarf am Juli 06, 2012, 11:46:11
I'm sorry to say that the latest beta does not terminate the way I run it, but 1.2.29 does work ok.
...I just had the chance to test my setup with the latest beta 1.2.32 (did not test with 1.2.29) but
it works for me now ;D
I noticed two effects though:
- after the run, TS-Doctor is listed in the RessourceMonitor as "abgebrochen" (?"canceled"?) instead of "beendet" (?"ended"?)
until it finally vanishes from the list....however the calling service seems not to be affected in my case.
- when TSDoctor calls comskip, TSDoctor uses approx 30% CPU Time (while comskip at the same momenbt uses 20%).
...that's a lot of CPUPower waisted for waiting on comskip to finish, one might say.
I'm still seeing it fail to run/close under the latest versions. I seem to have found when I run it through normally I sometimes see a pcr out of range message 'try to align' which I wonder is what doesn't get answered in silent mode? If you say Yes, it wants to save down as separate file which would then not get answered.
Just following up on this one again. I'm still seeing issues where it won't close properly after completing writing to the fixed file. I've installed the latest version so I can rule out any recent changes. Any ideas what I can try next.
Oh and I notice this only seems to be an issue for x264 encoded transport streams. Normal DVB mpegs are fine.