Cypheros Transportstream Forum

English-speaking Support => TS-Doctor 2.x => Thema gestartet von: Nicolas58 am Januar 14, 2018, 16:23:53

Titel: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 14, 2018, 16:23:53
Hello,

I'm trying the trial version of Ts Doctor, and I'll probably buy it, but I have some issues, and one of them is critical for me.

Context :
I have 2 Xeobox HDT 7810 (DTT recorders), one into my dining room and one into my sitting room. They are both connected to a NAS so I can share recordings between them.
The TV of my sitting room is a Samsung UE40F6320 and it is able to read those recordings (by UPnp), but only if they are "reformated" by TS Doctor (I don't understand why, since I keep ts extension and original encodings). The advantage of reading my recordings directly onto the Samsung TV is that it can decode 5.1 sound streams and send it to my 5.1 sound system, while the Xeobox records all streams but its reading output is only 2.0 whatever is the recording.
So my usage of Ts Doctor would be to cut advertisings, and make them readable by both my Xeobox and my TV. I don't want to change anything to encoding types.
The signal of my antenna is very good since I have changed the position of my antenna one year ago (I have only a few warnings and/or errors from time to time when some motorcycles go through my street). So I have done tests with very good recordings and with old recordings which have a bad quality.
The results seems to be identical in both situations.

Critical issue :
After having cut advertisings with TS Doctor, I cannot navigate trough some recordings with my Xeobox : the total duration displayed is less than the real duration, so the navigation cursor cannot go further than this false end time. If I start viewing from an accessible point, playing can continue further the false end time, but the timer stay fixed on the value of the false end time, and if I try to navigate it comes back immediately at the false end time. There is not this issue onto my original recordings (but I have deleted some of them before I discovered this issue...).
I have similar issues onto the Samsung TV with some of these recordings.
Maybe it is important : I answered yes to these questions : "Paths video PES length ?" and "FORTIS FTRI packet detected. Copy FTRI packed to new file ?"
I didn't see any other questions (except about multi part recording).

Other problem :
On the Samsung TV, some recordings seem to have no sound. Actually I need change the audio language to get it back ! Example : I begin to view a record, it seems to have no sound. If I go into the language menu, I see that the first language is selected. I select the second language (audio description) and I have sound. I come back to the first language and I have the normal sound ! While the sound is not read, navigation also doesn't work normally (even for recordings which end time is correct).
This is not critical, but maybe it can help you to identify my problems...

Questions :
- I use only two buttons of the main window : "prepare cutting" and "save new file". As TS Doctor is supposed to repair recordings, I don't understand why when I run a check on a new file created with "save new file" button, it seems to have the same number of errors as the original. Did I miss something ? I thought TS doctor would have patched it, even if it means inserting blank video or sound.
- Do you have an English documentation ?

I have unchecked the option "remux AAC(LATM) to AAC(ADTS)", it changed nothing to my sound problems.
Maybe the options about timer correction would solve my timer problems, but I don't understand what they do ?

Best regards.

PS : If you can solve my timer problems, I promise I will buy your great software immediately !
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 15, 2018, 00:34:05
Hi, could you post the log of such a problem file?

TS-Doctor creates a log for any new file. Same filename but with file extension .log

Makes it easier to see the technical details.
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 15, 2018, 21:49:27
Hi,

The attached log file is one of which that have the timer issues on both my TV and my Xeobox.
On my Xeobox the false end time is at 10'57" and on my TV the "breaking" time is at 9'21".
I say breaking time on the Samsung TV because the displayed total time seems to be OK, but the navigation is not possible after this time.


Thanks for looking at my issue.
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 17, 2018, 00:40:29
Hmm, found a strange value:
Speed: 1,7 MBytes/sec

That's the slowest processing speed I ever saw. What recording medium do you use?

The bitrate is very very low for over 2 1/2 hours of a full HD recording.

I guess it's DVB-T, isn't it?

What channel?
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 17, 2018, 22:33:43
Hi,

The channel is NRJ12 on French DVB-T.

Yes it seems very strange to me too, but not for the same reason :
Speed: 1,7 MBytes/sec
Duration: 00:48:10

The real duration of the final file is not 00:48:10, but 2 hours and about 10 minutes ! If you look at cutting details, you will find it.
The file size is 4424MBytes.
So for me the average bitrate is
4424 MBytes / (130*60) s = 0,567MBytes / s = 4,537Mbits / s

To my mind it is matching with these datarates :
PID stream average bitrates
$00DC: 4,1 Mbps
$00E6: 124,9 Kbps
$00E7: 124,9 Kbps
$00E8: 94,5 Kbps
$00FA: 0 bps
$00FB: 16,9 Kbps


You said that it was a very low bitrate for HD recording, but it's nearly what I have with all my recordings :
The video stream bitrate is usually between 3 and 6 Mbps (= Mbits / s, no MBytes...).
And you can add about 125 kbps (kbits/s) for each language stream.

I checked a few files, the last 2 lines of the log files, with speed in MBytes / s and duration are always false values.

Out of curiosity, could you tell me what are the bitrates of DVB-T in Germany ? (or eventually DVB-S and / or DVB-C).

Best regards.
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 17, 2018, 22:57:57
There is a misunderstanding regarding the values.

Speed and duration has nothing to do with the bitrate. Duration is the time needed to process the file and the speed is the processing speed of TS-Doctor. In your case TS-Doctor needed more than 48 minutes to process the file.

Here the numbers of a German HDTV channel:

File size of source file: 8763916476
Duration of source file: 01:32:18.253

Speed: 30,0 MBytes/sec
Duration: 00:04:38

PID stream sizes
$1451:   4.849.667 KB
$1452:     137.691 KB
$1453:     137.691 KB
$1454:           0 KB
$1455:      14.142 KB
$1456:     317.747 KB

PID stream average bitrates
$1451: 7,2 Mbps
$1452: 203,7 Kbps
$1453: 203,7 Kbps
$1454: 0 bps
$1455: 20,9 Kbps
$1456: 470,0 Kbps


Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 18, 2018, 13:53:55
Hello,

Ok you have a bit higher bitrate in Germany if it is DVB-T.

About processing speed and duration : I firstly thought it was supposed to be TS-Doctor processing values, but in that case it is also completely false for all my files !
My real values would be approximatively like yours : I think that for my files which were about 4 GBytes, it took approximatively 2 minutes.
Anyway I'm absolutely sure it was not more than a few minutes for each recording. I was impressed by this fast processing.

Best regards.
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 19, 2018, 00:48:31
Zitat von: Nicolas58 am Januar 18, 2018, 13:53:55
Ok you have a bit higher bitrate in Germany if it is DVB-T.

No, it's DVB-S.

DVB-T is not comparable because DVB-T in Germany is not HD but SD with MPEG2 codec. At the moment we are moving to DVB-T2 with H.265, what is in HD but with a newer more effective codec (lower bitrate).
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 19, 2018, 11:27:03
Ok.
And for my timer and navigation issues, do you have any news ?
Would it help you if I give you one of my recordings ?

Best regards.
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 19, 2018, 14:47:23
Please check the following setting. Under Settings/Preferences/Correction Behavior make sure the "Timer correction" is set to "Patch every time".

If this is allready set, it would be very helpfull to have a sample here to analyze the details. We could offer fast FTP server access to upload a sample file.
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 19, 2018, 15:14:25
It was set to "For PCR wraps without questions [recommended]".

I'll try to make some new tries this week-end and I'll give you the results.

Thanks.
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 20, 2018, 17:37:24
Hello,

I tried to patch one of my "broken" recordings with the option "Always patch" for timer correction. I didn't keep my original file after the first cutting, so I did this second patching from the cut (and broken) recording.
I also tried a second time with the option Enhanced PCR check active (search depth 140 MBytes and 1 PCR scan factor). I don't know what does PCR means, but it seems to to be related to the timer, that's why I tried it.

The result was exactly the same : I can't navigate after about 10 minutes from the begining with my Xeobox device.
You can see the 2 log files in attachment.

Can I send you the ts file on your FTP server ? No problem for me to send you the whole file : my internet connection speed is about 300 Mbits/s (optical fiber).

Thank you.
Titel: Re: Nex user - some problems and questions
Beitrag von: tsduser am Januar 21, 2018, 15:17:19
Just a short sidenote regarding the execution speed irritation (without the intention of hi-jacking this thread, of course):

The reportedly low speed indication is NOT caused by a slow storage medium, but obviously by a fault in TS-Docor itself:
The time-span reported isn't counted from the beginning of the cut operation, but from the point in time when TSD exe was actually started :-)
You can easily reproduce this behaviour by starting the Doc (2.0.105 will do just fine), then wait an hour or so before further processing a recording (incl. opening that recording after the hour), and the log will then report that it took 1h plus the actually needed time to complete the cut. The media speed is then concluded from the 1h-plus time period.

This problem was evident for a while (don't know exactly, sorry), but I never regarded this a major bug. And I had (and still have) little hope that correcting this "timer" issue will finally allow the Doc to keep a file opened for further steps after finishing one operation :-)
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 21, 2018, 22:21:44
@Nicolas58: Sent you a PM

@tsduser: We will check that an fix it with the next release.
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Januar 27, 2018, 18:42:35
Hello,

Do you have any news since I have sent you my sample file ?

Best regards.
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Januar 29, 2018, 23:13:26
We fixed processing speed bug for the next release.

The sample has is OK and should work. It seams to be a problem with the Xeobox. This box is unknown here in Germany and I don't know the chipset and technical features. Some receivers have additional files beside the main .ts file. This files have an index of iframes or other details to make navigation work.
This index files have the same file name but another extension, like .dvr or .ts.sc or .ts.ap or something like that.

Also there are often two types of mediaplayers available. One to playback the own recordings and an additional general mediaplayer to playback external files.
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Februar 17, 2018, 13:09:07
Hello,

Sorry for my long absence, I didn't have time to make more tests.
I have purchased my licence a few days ago because my trying period was finished. So I hope we will find a solution...
Actually your software is the only simple solution I found and I think it is great, I think there is only a little problem to find and solve (maybe a bug, or maybe something I didn't configure as needed).

You're right when you say that my Xeobox has additional files (.idx and .rap). I can send you a sample if you want.
This is some points I noticed :
- original recordings are split into 4Go files, named .ts, .ts1, .ts2...
- if I remove .idx and .rap files, the recording is still readable but navigation works only on the first part, corresponding to the first ts file. The total time is the real total recorded time, and the player is able to go to the second file (.ts1) during reading or fast reading, but navigation (jumps with arrows on the time line) and displayed actual time stops at the end of the first ts file.

So OK these points could make me think you're right when you say it is a problem related to the Xeobox.
But the next points let me think there is something else :
- some of my recordings cut with TS Doctor works almost perfectly : navigation is ok on the whole recording, but the total displayed time is the total time of the original video !
- on the sample that I sent to you, navigation stops at about 10 minutes from the beginning, so it has nothing to see with the end of the first file of the original recording.
- this same sample has also navigation problems with my Samsung TV, at about 9 minutes from the beginning.

About the two types of mediaplayers often included in recorders :
In my case there are two ways to access medias :
- configure a directory which can be local (USB drive) or a network directory path (my NAS in my case). This place is used to store recordings, but it can have subdirectories and it allows to read every type of files supported by the player (several formats for videos, images and sounds). TS files are considered as recordings by the player (these files appear in the "recording list") and other formats are considered as imported files (these files appear in the "imported files list").
- use a DLNA server

I tried to convert some recordings into .mkv files with TS Doctor, so they are considered as imported files by the Xeobox : for some reason that I don't understand, the Xeobox is unable to get sound from these files. However it is able to get sound from some other .mkv files which are not created from TV recordings.

I tried to use the DLNA server of my NAS (Synology) : it is the only way that my Samsung TV uses. On some recordings the Samsung TV has navigation problems. I didn't see navigation problems with the Xeobox when using the DLNA server, but I have no sound on it (maybe the same problem as for .mkv files ???).

So as you can see, I despair of finding a simple solution always working... I think the simplest solution would be to make my recordings work on the Xeobox (it doesn't matter the Samsung TV doesn't read it), as I have a Xeobox at the 2 places where I want to read my recordings.

I am convinced that some erroneous time information remains in the files generated by TS Doctor, otherwise how could the reader display the total time of the original video in place of the total time after cutting ?
I made a new try yesterday, on this recording the total recorded time was 3 hours. After cutting I have 1 hour and 53 minutes of video. I asked TS Doctor to save the result into a completely new file name, and after that the Xeobox is always saying that the total time of the video is 3 hours !!! In this case the navigation works, it's only the total time that is false, but the Xeobox has not any possibility to know where does this new file come from. So the total time of 3 hours is inevitably written into it !

Maybe it has something to see with the Fortis IRTA packet ? It says this packet is detected and I asked to keep it... I'll make a try without keeping it.
Another idea : maybe the index file is the same as the one you can create with your tool "TS index creator" ? Should I try it and rename the resulting files into .rap and . idx ?

Maybe you could examine my .idx and my .rap files ?

There is a technical description of my Xeobox at this url : https://www.aervi.fr/Download/PDF/XEOFIX/hd_t_7810.pdf
The Xeofix company no longer exists, I bought my devices several years ago. Apart from the fact I cannot cut my recordings with it, I'm very satisfied of its quality for TV recording and playing. I is not adapted for playing any media formats, but it doesn't matter for me.

Thank you for your help...
Titel: Re: Nex user - some problems and questions
Beitrag von: Mam am Februar 17, 2018, 14:39:00
a very simple, but very powerful too, solution is to use a "real" Mediaplayer.
Those included in your Samsung TV and other devices are crippled to the bones. They only play those formats that is usually used in the shop of that company. Pure shit!

Get a simple PI (3), throw Libreelec onto its SD card, connect it to LAN and TV (with HDMI) and fire it up!

It plays almost everything that is currently available (no UHD / H265 stuff yet!), and it silently converts those input files to an output format, your TV Set does not start to choke on!

You can get all needed parts for ~50€ (a little bit more if you want a fancy housing too), there are also vendors which offer them "ready built" (but beware, some of them include "doubtful" addons which are not supported by the main creators, better play it safe and use only the free downloadable stuff from the original distribution websites).

(its so easy to be used, even wifes can be trained to use them alone and without too many or too loud complains! *)

(*) may take some time for training  ;D
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Februar 17, 2018, 19:32:20
Thank you for your answer Mam. I already thought about this solution as a "last chance solution". It is not my preferred solution because I don't want to multiply the number of devices and remote controls (libreElec requires a smartphone or a keyboard and/or a mouse, it is another type of remote control...). As you say, only one device is also only one way to learn for my wife and my children, who often schedule recordings themselves and watch them directly on the Xeobox without having it cut by TS Doctor (I will cut only good films that I want to keep...).

I agree with you, I am tired of all these commercial devices (TV's, media players and recording boxes) that often do only half of what they promise, and what they promise is already half of what should be possible... Of course I'm not talking about your PI based solution as it is open source.
But using the same device for recording and reading should not be mission impossible... I'm always hoping for a solution. If one day we have a Raspberry Pi with a DVB-T card, surely I'll be very interested in it.
Today I have my 2 Xeobox devices and I'd like to use them at their potential...

And with this great software - TS Doctor, I have almost a perfect solution to use ONLY my Xeobox devices.
So if the author of this software is able to and has the kindness to help me make it work, I prefer this solution very much.
Maybe we will find why the total time of the original recording remains in the cut file, and maybe he will correct some bug at this occasion, it would be a win for everyone !

Note that the readers of my TV's are not so bad : today I finally found a way to jump through the breaking point on my Samsung TV (there are 2 navigation bars, the first one is displayed when I directly use right and left buttons and this one doesn't jump through the breaking point, but there is another one accessible into some menu, and this one works).
And  my other TV is an old Sony KDL 32CX520, its network functions are limited so I had disconnected it from the network a long time ago, but I finally reconnected it today and it reads my cut recordings like a charm ! The only issue with the Sony TV is that I didn't find how to choose the language. But It seems to choose the correct one (French) on all videos, whereas I have to change it on the Samsung TV.

So I have found a work around, but if Cypheros agrees to work on the problem it would help my wife and my children, used to the Xeobox menus, and I would be very grateful...
Titel: Re: Nex user - some problems and questions
Beitrag von: Cypheros am Februar 17, 2018, 22:14:33
The indexer for Enigma ist only possible because Enigma is "Open Source" and you can look into the source code to see how the indexing is done.

I can't find Xeofix source code or any documentation explaining the format of the index files.
Titel: Re: Nex user - some problems and questions
Beitrag von: Mam am Februar 18, 2018, 07:50:40
aah, I see, you did not really understand my strategy behind my suggestion (PI & Co), so lemme try it again:

* Many Vendors use "special" Formats that are not documented anywhere like your Xeofix for example. Even between different machines from the same company, format changes are not uncommon. Many (most) companies block requests for documentation, if a company is already out of business, its unlikely that you ever manage to find somebody who can tell.
* of course, its normal, that you can play back a recording on the same device where you have done the recording (at least, this SHOULD be very very normal)
* if you edit the recordings with external tools like the TSDoc it is also common that afterwards some special informations that are needed by that particular device are missing, the resulting file wont play there anymore.

This happens almost EVERYWHERE and ANYTIME.

You can either overcome this by making the tools more clever and producing more specialized output files for each possible device (as one can easily see, this is a job for somebody that wants to spend his lifetime on chasing new specials that come out each day and anywhere in the world),
OR you switch your workflow to a more submissing, but always working, way: use the tools to convert the files into a common and widespread format that can be played EVERYWHERE.

As you can guess, the last part is my way to go.

I take recordings from any device I have (and I do have some different, you can bet), the TSDoc can read them, so I use him for cutting/cleaning.
The output file is then put into an ecoding tool (I use "Handbrake", others prefer VideoReDo or so) that recreates the whole file, changing video AND audio formats.
My standard output format is currently 720p25 with AAC2.0 sound (or Dolby Digital 5.1 if the source is multichannel) plus subtitles, and all my players (if PIs, or PCs, or even builtin Players in TVs) are capable of playing back THIS format. (the H264 video settings are rather "simple", so even small devices can handle them without stress).
These settings are stored as "Presets" into "Handbrake", so I dont need to pick anything for the next file to do...

I can just drop these (final) output files into my central film store in my LAN and dont have to worry where, if and by what it can be played.

Therefor I dont need to chase "special" things of some devices, I can even change the recording device as long, as the new one's format can be read by the TSDoc, without affecting anything in my workflow.

And for the "wife's training" its also nice to present them the same user interface at the players everywhere. So here on my PIs, Fire-TVs, Smart-TVs or PCs, they will always find the same Version of KODI, with the same settings everywhere (and using a central database too). All players look the same and show the same menus and files. I just copy a new film in and use a tool called "Ember" to rename it correctly and add the infos/pictures from the internet. All players do present these infos and allow them to be searched (wife's training #234: "how to find a certain movie among the many thousands that are stored").

Anyway, all this needs some disciplin and careful planning ahead...  ;D

(and of course, your demands may be totally different, so just take this as an example of a working workflow and create your own)
Titel: Re: Nex user - some problems and questions
Beitrag von: Nicolas58 am Februar 18, 2018, 18:11:45
Ok Mam, I understand your point of view, and your strategy is obviously a good strategy given that you have numerous recorders and the possibility to put Kodi on all your readers.
At this time my goal is to be able to read my recordings with the devices I already have, and to keep them in a standard format with the least amount of re-encoding possible (re-encoding is always a loss of quality and needs more processing time...).

Before this conversation I didn't know that almost all recorders were adding their own information in their own format. For free DVB I thought stupidly that they wrote the whole DVB stream without changing anything, except those which encrypt the stream (in France, some internet providers do that with their box...).

In the case of my Xeobox, it remains true that it is generally able to read the stream after cutting the file with TS Doctor, so without the additional .rap and .idx files. On original recordings it seems that the .idx file is necessary to go from .ts to .ts1 file (but with TS Doctor I merge them into one file), and removing the .rap file seems not to be a problem. The issue of the breaking point for navigation is only on some recordings, and all cut recordings have a false total time information after cutting. And this false time information is necessarily written in the .ts file.
I firstly thought that this bad time information was in a standard format, but now if Cypheros is sure of its PCR time correction (sorry if it is not the good name, I saw it into the options...), I understand it could eventually be in a special substream of the file and in a special format. And I'm not sure that this false total time is the origin of the issue.

Cypheros, of course I understand you can refuse to spend time examining some files which are not standard.
I'm a software developer in a totally different field, so I don't know much about video streams (but I know how long can be the research of an issue...), and I'm simply wondering if you could eventually demux the streams and try to find where this bad time information is hiding. TS Doctor's demuxer finds only video, audio and subtitles streams, maybe something else is hiding in the whole file ?
Fortis FTRI packets are present in the file and if I do not ask to keep them the file is not readable by the Xeobox. I don't know what are these Fortis packets, but something tells me that the Xeobox is a derivative of a family of products ("Fortis family") that you know and you have added special processing for them.
So maybe you could discover something new about this family of products ?

Again, if you refuse I can understand. Your software remains a very good software and I'm satisfied of it !

I have other minor issues related to TS Doctor, I will post it in another topic if I have time for that.