Register Forum Rules FAQ Members List Social Groups Search Today's Posts Mark Forums Read

 AnimeSuki Forum [Encoding Technobabble] Ordered Chapters And You - what, how, why, when

 2007-12-16, 21:43 Link #2 Eviltape Senior Member     Join Date: Oct 2007 Location: Florida, USA I cry tears of joy for you; I can finally slice off those damn 1:30 OPs and EDs of series that graze just past the 4.7GB mark after my custom XviD to h264 encodes and I can now select which one I like the best. Many thanks, yet again. This is a great technique to (attempt to) master to learn how to compress entire series rips and their OPs and credits/EDs: Hollywood, Anime, Bollywood, and otherwise. DVD Rippers, take note.
 2007-12-16, 22:39 Link #3 Nicholi King of Hosers   Join Date: Dec 2005 Age: 34 TheFluff has stolen my sekrats!!! SEKRATS!!!
 2007-12-17, 19:40 Link #4 Zero1 Two bit encoder Fansubber     Join Date: Jan 2006 Location: Chesterfield, UK Age: 33 Good to see you longposting. Not so good to see you trolling/taking pot shots. Anyway, information is a great thing. Now go and teach the raw cappers not to suck, and the raw providers to spot shitty upscales and awarpsharp abuse. I like the idea of these chapters and how you can effectively remove so much redundancy. It's also good because when I've seen an Op once/twice, I generally don't want to see it again (unless it's very good). There's also the added bonus of having the Op/Ed seperately if you do happen to like them (I have a few that I extracted and encoded, such as Gundam X's Op). Oh that's something. In a series that has been linked to an external Op/Ed; what behaviour would you get if that Op/Ed didn't exist? Would it just play the episode regardless? That would be pretty handy if a series had an Op you didn't care for. Just leave those out of the batch torrent and watch each episode without having to skip/seek/go to chapter. Also this has no practical reasoning, just pure curiosity. I'm guessing the answer to these are yes, having some knowledge of how splitters and decoders work, but I'll ask anyway. What would happen if the Op/Ed were different framerates to the original episode (eg the episodes are 23.976 and the Op/Ed is 29.97) (lol poor mans VFR); would it still play? Also what if different resolutions or codec? (maybe for example you found a textless Op/Ed and replaced the original with those, but they were encoded slightly differently)? It probably depends on the player/codecs/splitter as much as anything, kind of like switching to different angles using different codecs maybe. Anyway, if nothing it might be something to test to see if you can get it to break anything decoder side, or maybe serve as another crazy test file for CCCP. __________________ Made of win and awesome: 7-zip // x264 (Sharktooth's builds) // XviD (Koepi's builds) // MP4box (celtic_druid's builds) // Firefox // CCCP // CoreAVC
TheFluff
Excessively jovial fellow

Join Date: Dec 2005
Location: ISDB-T
Age: 31
Quote:
 Originally Posted by Zero1 Good to see you longposting.
I learned from the best!

Quote:
 Originally Posted by Zero1 Not so good to see you trolling/taking pot shots.
No post is truly complete without a touch of trolling to make it interesting. :V

Quote:
 Originally Posted by Zero1 Now go and teach the raw cappers not to suck, and the raw providers to spot shitty upscales and awarpsharp abuse.
I'd have to learn to write technobabble in Japanese first, so maybe in 10 years or so (given how long it took me to get where I am with English) but even then I'm afraid it's a lost cause...

Quote:
 Originally Posted by Zero1 Oh that's something. In a series that has been linked to an external Op/Ed; what behaviour would you get if that Op/Ed didn't exist? Would it just play the episode regardless? That would be pretty handy if a series had an Op you didn't care for. Just leave those out of the batch torrent and watch each episode without having to skip/seek/go to chapter.
Yes, that's what happens if you use Haali's splitter. If it can't find a segment, that part of the timeline is just skipped as if it didn't exist.

Quote:
 Originally Posted by Zero1 Also this has no practical reasoning, just pure curiosity. I'm guessing the answer to these are yes, having some knowledge of how splitters and decoders work, but I'll ask anyway. What would happen if the Op/Ed were different framerates to the original episode (eg the episodes are 23.976 and the Op/Ed is 29.97) (lol poor mans VFR); would it still play? Also what if different resolutions or codec? (maybe for example you found a textless Op/Ed and replaced the original with those, but they were encoded slightly differently)?
All linked segments must have the same set of tracks with the same set of codecs; additionally the CodecPrivate header (and some other parameters) must be (mostly?) the same. Internally ordered chapters are just implemented with seeking so to avoid confusing the playback environment a lot you can't switch things like codecs, resolutions, samplerates or number of channels mid-playback. Different framerates and different display AR's work fine though.

(Reference for the following paragraphs: http://haali.cs.msu.ru/mkv/codecs.pdf)
One interesting gotcha with this is that for S_TEXT/ASS and S_TEXT/SSA the entire script info and styles sections of the file is stored in the CodecPrivate header, so any style you want to involve in segment linking of any kind must exist in all the involved segments; the script headers need to be the same as well. I have not yet tested how far you can stretch this and still get away with working playback.

For V_MS/VFW/FOURCC (pretty much anything muxed from AVI), the CodecPrivate is the BITMAPINFOHEADER of the AVI file (contains stuff like resolution and bits per pixel) followed by some codec-specific private data, so your mileage may vary with which settings you can change and which you can't. In some cases mkvmerge can exit with warnings but the resulting file plays anyway.

For V_MPEG4/ISO/AVC (h.264) the CodecPrivate is a rather interesting structure that among other things stores the profile, the level, the SPS's and PPS's.

For anything else, just read Haali's PDF or experiment. For the most part it has been pretty simple to get things working right in my tests.

Quote:
 Originally Posted by Zero1 It probably depends on the player/codecs/splitter as much as anything, kind of like switching to different angles using different codecs maybe. Anyway, if nothing it might be something to test to see if you can get it to break anything decoder side, or maybe serve as another crazy test file for CCCP.
One of these days I'll prepare some samples, both for testing and for getting people to understand how awesome these features really are. :V
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read

Last edited by TheFluff; 2007-12-17 at 20:46.

 2007-12-18, 14:12 Link #6 jfs Aegisub dev     Join Date: Sep 2004 Location: Stockholm, Sweden Age: 33 One important note: Two Vorbis streams, even encoded with the same encoder and same settings, might have different CodecPrivate headers, because it AFAIK stores the codebook, which is dynamic for Vorbis. At least that's something I think I noticed once when i tried concatenating two Vorbis streams in mmg. __________________ Aegisub developer [ Forum | Manual | Feature requests | Bug reports | IRC ] Don't ask for: More VSFilter changes (I won't), karaoke effects, help in PM's
TheFluff
Excessively jovial fellow

Join Date: Dec 2005
Location: ISDB-T
Age: 31
Well after consulting the literature I must conclude that I may be behind my times by almost two years. Quoting the following mkvtoolnix changelog item from 2006-02-27:
Quote:
 Originally Posted by mkvtoolnix changelog * mkvmerge: enhancement: mkvmerge will no longer refuse to concatenate files with differing Codec Private contents and only issue a warning in such cases.
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read

 2007-12-20, 19:17 Link #8 TheFluff Excessively jovial fellow     Join Date: Dec 2005 Location: ISDB-T Age: 31 Ugh, double post, but anyway, I created a sample that demonstrates some of what you can do with ordered chapters. Download here: http://uppcon.se/thefluff/samples/or...ters-sample.7z (30mb) Edit: "cheat sheet", i.e. the XML files I used: http://uppcon.se/thefluff/samples/or...sample-xmls.7z __________________ | ffmpegsource 17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's. 17:43:16 <~deculture> nice job, fag! 01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read Last edited by TheFluff; 2007-12-24 at 01:12.
 2007-12-24, 15:54 Link #9 Jubalint Timer and Stuff     Join Date: Oct 2006 Location: Meh, who cares? I was exporting the xml chapter files from Hitsuji's encodes of Suzumiya and looking at those to figure out ordered chapters along with mkvtoolnix's (sparse) documentation on UID's, but this makes things a whole lot easier! Especially some of the applications of it you suggest. Thanks Fluff, your awesome!
 2008-01-01, 08:31 Link #10 4as ForUs     Join Date: Sep 2006 Location: Poland Age: 32 Hey TheFluff, please help ;0 Im using ordered chapters to add an ending from a diffrent file. Original anime had already been writen in mkv with ordered chapters, so it was only the matter of changing the ChapterSegmentUID... or so I thought. The original part: Code:  3611616 0 1 00:12:44.514000000 00:23:11.174000000 b8 7c 13 a0 3d 65 ee 07 82 53 d4 84 5f 4c a6 74 Part B und 453616 0 1 00:23:11.174000000 00:24:15.963000000 b8 7c 13 a0 3d 65 ee 07 82 53 d4 84 5f 4c a6 74 Ending Credits und 51546130566 0 1 00:24:32.619000000 00:24:48.936000000 b8 7c 13 a0 3d 65 ee 07 82 53 d4 84 5f 4c a6 74 Next Episode Preview und  And my change: Code:  3611616 0 1 00:12:44.514000000 00:23:11.174000000 b8 7c 13 a0 3d 65 ee 07 82 53 d4 84 5f 4c a6 74 Part B und 453616 0 1 00:00:00.000000000 00:01:04.531000000 80 83 73 28 8b ea 1f 5d ab 7d ed 98 97 4f ca 87 Ending Credits und 51546130566 0 1 00:24:32.619000000 00:24:48.936000000 b8 7c 13 a0 3d 65 ee 07 82 53 d4 84 5f 4c a6 74 Next Episode Preview und  I figured out the ChapterTimeEnd by looking at "Duration" from the mkvinfo. It works as intended, but there are about 10mins of "empty space" on timeline, between the ending credits and next episode preview. When played, from beginning, the video player simply skips that space, but when I move playhead there, it eather crashes or only shows last rendered frame. I've read your "rather lengthy guide" and found "...remember that all ChapterTimeStart/End’s refer to timestamps of the underlying segment...", would that have something to do with my problem? Im not really sure what to do with it - I've loaded timecodes into aegisub, but dont know what to do next with it. __________________ Last edited by 4as; 2008-01-07 at 10:14.
 2008-01-06, 20:59 Link #11 TheFluff Excessively jovial fellow     Join Date: Dec 2005 Location: ISDB-T Age: 31 Sorry for being late with the response, I've been gone over the holidays. Anyway, that looks correct to me (at a glance, at least). How did you remux it? With GDSmux? Remember that mkvmerge always changes the SUID when remuxing so if you remuxed the original file with it, the SUID pointing at that will now be invalid. __________________ | ffmpegsource 17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's. 17:43:16 <~deculture> nice job, fag! 01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
 2008-01-06, 21:48 Link #12 4as ForUs     Join Date: Sep 2006 Location: Poland Age: 32 With mkvmerge, and yes it changed SUID. But as I understand, if it was wrong, the ending would not show at all, but it plays fine. Also I did some "research"; it seems that if I move the file (mkv with ending) to another folder, it obviously doesnt find it, and the gap (along with the ending) is gone. So something may be wrong with the ending, but no matter how I look it is 1min long, without any 10+ mins gap :S __________________
 2008-01-07, 08:11 Link #13 Phar Junior Member   Join Date: Jul 2007 You made a notation error: 0:01:04.531000000 Should be 00:01:04.531000000 I figure the splitter just reads the numbers and ignores the colons, so it was reading 00:10:45.
4as
ForUs

Join Date: Sep 2006
Location: Poland
Age: 32
Quote:
 Originally Posted by Phar You made a notation error: 0:01:04.531000000 Should be 00:01:04.531000000 I figure the splitter just reads the numbers and ignores the colons, so it was reading 00:10:45.
Hmmm... I dont know how it got there, but in my actual code it is correctly written as “00" (probably deleted it while adding the bold tag).
Just to be sure, I remuxed it again, and yeah, the gap is still there.
Im in dead end here ._.
I have no idea whats wrong...

EDIT:
This got me thinking, so I tried and changed the ChapterTimeEnd to simple 00:00:04.000000000. I presume ending should run now only for 4seconds, but surprisingly enough, it still played the whole thing… it is like ChapterTimeEnd doesn’t do anything o_O;
__________________

Last edited by 4as; 2008-01-07 at 11:11.

 2008-01-07, 11:35 Link #15 TheFluff Excessively jovial fellow     Join Date: Dec 2005 Location: ISDB-T Age: 31 You're obviously doing it wrong SOMEWHERE but I can't figure out exactly what it is. I'm not sure what exactly you're trying to do either; what exactly is the file you're remuxing to going to contain and what is going to be linked in from other files? __________________ | ffmpegsource 17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's. 17:43:16 <~deculture> nice job, fag! 01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
 2008-01-07, 12:36 Link #16 Phar Junior Member   Join Date: Jul 2007 I think I've managed to reproduce the problem. Make absolutely sure every ChapterUID is unique. It seems if two chapters have identical IDs, it simply takes the time of the longest of the two and assigns it to the first to appear (the remaining time is filled with blackness). The second chapter is dropped. Check the chapter menu to see which is missing; that would be the one with the duplicate UID (Note that, for instance, 0114 and 114 are considered the same UID.)
4as
ForUs

Join Date: Sep 2006
Location: Poland
Age: 32
Quote:
 Originally Posted by Phar I think I've managed to reproduce the problem. Make absolutely sure every ChapterUID is unique. It seems if two chapters have identical IDs, it simply takes the time of the longest of the two and assigns it to the first to appear (the remaining time is filled with blackness). The second chapter is dropped. Check the chapter menu to see which is missing; that would be the one with the duplicate UID (Note that, for instance, 0114 and 114 are considered the same UID.)
You know what?
YOU KNOW WHAT?!
...
IT WORKED
Instead of those random numbers like 10130428 and 30130428; I putted, for first 6 chapters, numbers from 1 to 6… and it worked!
Seems like original file had wrong UIDs, as I didnt touch them.
Anyway, Phar, your a life savior! Thank You Thank You Thank You.
Also, thank you TheFluff.
It is great to see there there are still peaople willing to help.

PS: Original UIDs looked pretty unique to me (and no chapter was missing in chapter menu)… I cant think of nothing else then those UIDs where TOO complicated, seeing how simpler ones fixed the problem :S
__________________

 2008-01-07, 19:53 Link #18 Nicholi King of Hosers   Join Date: Dec 2005 Age: 34 It's always the UIDs... never would have looked at the chapter ones though . /me shakes fist and lols that encoders have taken over chat so ScR3WiEuS has nothing to say! Here is an extra tidbit TheFluff hasn't updated to his page yet. Simple windows script from Haali to grab the segment UIDs of files you are linking to, useful if you are remuxing lots and hate re-writing those segment uid's everytime. You run it as Code: cscript //nologo path\to\updateseg.js \path\to\chapters.xml \path\to\updated_chapters.xml In the chapters file you write in @@path\to\file.mkv@@ for ChapterSegmentUID. If you can't run the script it just means you need to update Windows Script Host.
 2009-06-05, 11:04 Link #19 Jimmy C Senior Member     Join Date: Jun 2007 Pardon me for reviving an old discussion, but I've just tried ordered chapters after reading this. I have to say I'm impressed. I want to ask one thing: I can see that the individual tracks in an MKV also have their own track UIDs, is it possible to create an Edition that takes video from one time period and play it with the audio from a different time period? For example, texted vs textless OPs and EDs. Even with ordered chapters to select the proper segments, I'll still have to include the audio for those twice. But if it's possible to assemble a "virtual segment" I can just instruct it to take the alternate video, but continue playing the default audio. So, is this possible?
 2009-06-05, 14:54 Link #20 TheFluff Excessively jovial fellow     Join Date: Dec 2005 Location: ISDB-T Age: 31 No, unfortunately not. You can only grab the entire timeline (video, audio, subtitles) at once; you cannot pick and choose what tracks you want to grab. __________________ | ffmpegsource 17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's. 17:43:16 <~deculture> nice job, fag! 01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read