Release patching

[Starks]

Over the years I've seen some groups giving leechers tools to patch a release in place of or in addition to the issuing of a v2.

Can someone please explain this approach to me?

[xris]

Originally Posted by Starks

Can someone please explain this approach to me?

Are you asking for the rational of this approach or the practicalities? It's not clear if you are asking "why" is it done or "how" it is done.

[pichu]

Originally Posted by xris

Are you asking for the rational of this approach or the practicalities? It's not clear if you are asking "why" is it done or "how" it is done.

It looks like "how." They provide a patch, and you patch them through some opensource software. Every groups are different, so they tend to be different from one another, and thus the method to patch the files may vary from groups to groups. Need to see examples first.

[TheFluff]

If you're asking for the why, distributing a small (couple megabytes at most) patch is a lot easier than redistributing the entire file.

If you're asking for the how, see this handy guide: http://www.srsfkn.biz/2008/11/11/mak...-with-xdelta3/
(xdelta3 is probably the best program for this; it works on both windows and unix, is easy to use and the executable size is small. You can of course use mkvmerge instead but it's a bit more annoying.)

[TGEN]

Originally Posted by TheFluff

If you're asking for the why, distributing a small (couple megabytes at most) patch is a lot easier than redistributing the entire file.

If you're asking for the how, see this handy guide: http://www.srsfkn.biz/2008/11/11/mak...-with-xdelta3/
(xdelta3 is probably the best program for this; it works on both windows and unix, is easy to use and the executable size is small. You can of course use mkvmerge instead but it's a bit more annoying.)

Also, it gives guarantees as to what the resulting file will be like, unlike merging in a new script which will result in a different file for everyone (there's only an infinitesimally small chance of a similar file being produced with mkvmerge).

[False Dawn]

Originally Posted by TGEN

(there's only an infinitesimally small chance of a similar file being produced with mkvmerge).

Oh? Why's that? I always assumed that if I mkvmerge the same components into an mkv as someone else on a different computer, then we'd get the same results. Why isn't this the case? Just differences in settings, things like that?

[TheFluff]

Originally Posted by False Dawn

Oh? Why's that? I always assumed that if I mkvmerge the same components into an mkv as someone else on a different computer, then we'd get the same results. Why isn't this the case? Just differences in settings, things like that?

Because there are things (like various unique identifiers (segment UID's, track UID's, attachment UID's etc etc) and other stuff) in the file structure that are chosen pseudorandomly upon muxing. The version of the muxing application is also stored, and IIRC the time/date of the muxing start is stored too. Which means the CRC will change every time you remux.

Of course, given the same version of the muxing application and control of the muxing settings (via a bat file), the file should always be functionally identical, even if the CRC isn't the same.


edit: obviously this works best for softsubbed stuff. If you need to change something hardsubbed, you should not reencode the entire file because that will change the coding in the entire file and the patch will have to diff the entire video, so it'll be huge. Instead just reencode the scene where the offending hardsubbed stuff is and do some cutting and splicing.

[TGEN]

Originally Posted by TheFluff

edit: obviously this works best for softsubbed stuff. If you need to change something hardsubbed, you should not reencode the entire file because that will change the coding in the entire file and the patch will have to diff the entire video, so it'll be huge. Instead just reencode the scene where the offending hardsubbed stuff is and do some cutting and splicing.

Wildly off-topic, but I wonder what that would do to forward references just before the splice and backward references just after. Would the cutting/splicing tool account for that properly?

[sneaker]

Ureshii uses mkvmerge to patch the first episode of Kurozuka and the resulting file is 100% bit identical on all PCs.

Here's the batch they're using (they provide a binary of mkvmerge and the corrected .ass file with it):

@echo off
SET PATH=""
:start
echo Patching [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv....
if not exist [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv goto notfound
>NUL "bin\mkvmerge" -o "[Ureshii]_Kurozuka_-_01v2_[AAC-H264-720p][F76DF54F].mkv"  --language 1:jpn --track-name "1:Kurozuka 01: Adachigahara" --default-track 1:yes --display-dimensions 1:1280x720 --language 2:jpn --track-name "2:2.0 LC-AAC" --default-track 2:yes -a 2 -d 1 -S "[Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv" --language 0:eng --track-name "0:Styled ASS" --default-track 0:yes -s 0 -D -A "kurozuka01[FINAL]v4.ass" --track-order 0:1,0:2,1:0 --title "Kurozuka 01: Adachigahara" --engage no_variable_data
goto end
:notfound
echo -----
echo FILE NOT FOUND!!
echo Please copy your "[Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv" with the 
echo ORIGINAL FILENAME to this folder an run this patch...
pause
goto start
:end
echo DONE!
echo Deleting v1 file...
echo -----
del /f /p [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv

They even put the correct crc32 of the output file into the output filename and they're using ordered chapters.

[martino]

Originally Posted by TheFluff

The version of the muxing application is also stored, and IIRC the time/date of the muxing start is stored too. Which means the CRC will change every time you remux.

--engage no_variable_data

will not store the time of muxing (as well as any other data that can change from one mux to another) and thus produce the same CRC if muxed with the same version (as you mentioned).

EDIT: beaten by sneaker... kinda.

I was always wondering though whether the same mkvmerge version across different platforms would produce the same CRC... lazy~

Anyway, I'd just like to add that in the past patching through a bat file and mkvmerge used to be more common than xdelta3. Seems to be the other way round these days... actually, I don't remember seeing an mkvmerge patch in a while now.

Personally I think it's for the better, cross-platform patching is more reasonable this way (or at least I think; only if we could see some bash scripts along those batch ones, not like it's much copy-paste work for the end user in the end so just a matter of effort), although I've had some weird problems with an xdelta3 patch made under win32 and applied under *nix. Who knows what that was...

Still, the amount of patches around and v2s is in a rather sad ratio (mainly when softsubbed MKVs are concerned). It sure would be nice to see more of them around in the future. Even though, who is to complain with BT around?

[TheFluff]

Originally Posted by TGEN

Wildly off-topic, but I wonder what that would do to forward references just before the splice and backward references just after. Would the cutting/splicing tool account for that properly?

depends on what cutting/splicing tool you use, if you use mkvmerge you can only cut at IDR-frames so the problem solves itself.

I had forgotten about --engage no_variable_data, but xdelta3 also has the added advantage of being a lot easier to use for unix users that can't use the .bat file. Then again you could just provide a shell script too.

[TGEN]

Originally Posted by martino

only if we could see some bash scripts along those batch ones

Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!

[edogawaconan]

Originally Posted by TGEN

Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!

is it even possible to make sanely sized & statically linked mkvmerge?
Irrelevant if using xdelta

[TGEN]

Originally Posted by edogawaconan

is it even possible to make sanely sized & statically linked mkvmerge?
Irrelevant if using xdelta

Distributing binaries along with it is a bad idea, as not everybody is running Obesux.

MKVMERGE=`env mkvmerge`
if test "x$MKVMERGE" = "x"; then
        echo "go cowpile mkvmerge for your platform"
else
        $MKVMERGE foo bar baz quux etc
fi

[checkers]

Originally Posted by TGEN

Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!

Fansubbers don't care about Mac compatibility, why would they care about NetBSD 1 compatibility?

[Starks]

Originally Posted by TheFluff

depends on what cutting/splicing tool you use, if you use mkvmerge you can only cut at IDR-frames so the problem solves itself.

I had forgotten about --engage no_variable_data, but xdelta3 also has the added advantage of being a lot easier to use for unix users that can't use the .bat file. Then again you could just provide a shell script too.

xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>

[edogawaconan]

Originally Posted by Starks

xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>

> cd /usr/ports/misc/xdelta3/ && sudo make install clean

> sudo apt-get install xdelta3

(I don't know how to install it but it exist on Arch)

etc

[TGEN]

Originally Posted by checkers

Fansubbers don't care about Mac compatibility, why would they care about NetBSD 1 compatibility?

Hey, the oldest NetBSD I have running is a 1.4.2 (VAX even!), no need to exaggerate...

[Nicholi]

/me approves of xdelta3 patching

It is certainly easier to compile than mkvtoolnix at least, lol. Could be packaged with a script as well for easy newb use. Also would allow you to fix more than JUST script errors, since you would be able to make a binary diff for literally anything (even re-encoded video segments). Forgotten fonts, chapters, tracks, other attachments, anything really.

[TheFluff]

Originally Posted by Starks

xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>

as edogawaconan pointed out, it's a unix program that has been ported to windows, not the other way around