AnimeSuki Forums

Register Forum Rules FAQ Members List Social Groups Search Today's Posts Mark Forums Read

Go Back   AnimeSuki Forum > Anime Related Topics > Fansub Groups

Notices

Reply
 
Thread Tools
Old 2009-01-07, 10:08   Link #1
Starks
I see what you did there!
*Scanlator
 
 
Join Date: Apr 2004
Age: 26
Send a message via AIM to Starks
Release patching

Over the years I've seen some groups giving leechers tools to patch a release in place of or in addition to the issuing of a v2.

Can someone please explain this approach to me?
__________________
Starks is offline   Reply With Quote
Old 2009-01-07, 10:20   Link #2
xris
Just call me Ojisan
 
 
Join Date: Jan 2003
Location: U.K. Hampshire
Quote:
Originally Posted by Starks View Post
Can someone please explain this approach to me?
Are you asking for the rational of this approach or the practicalities? It's not clear if you are asking "why" is it done or "how" it is done.
xris is offline   Reply With Quote
Old 2009-01-07, 11:21   Link #3
pichu
Senior Member
*Fansubber
 
 
Join Date: Jul 2004
Quote:
Originally Posted by xris View Post
Are you asking for the rational of this approach or the practicalities? It's not clear if you are asking "why" is it done or "how" it is done.
It looks like "how." They provide a patch, and you patch them through some opensource software. Every groups are different, so they tend to be different from one another, and thus the method to patch the files may vary from groups to groups. Need to see examples first.
__________________
A small tool I made, and Learn how to make fansub karaoke in After-Effects: AFXKRK
pichu is offline   Reply With Quote
Old 2009-01-07, 13:17   Link #4
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 27
If you're asking for the why, distributing a small (couple megabytes at most) patch is a lot easier than redistributing the entire file.

If you're asking for the how, see this handy guide: http://www.srsfkn.biz/2008/11/11/mak...-with-xdelta3/
(xdelta3 is probably the best program for this; it works on both windows and unix, is easy to use and the executable size is small. You can of course use mkvmerge instead but it's a bit more annoying.)
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Old 2009-01-07, 14:12   Link #5
TGEN
Far out, man!
*Fansubber
 
Join Date: Jul 2008
Age: 30
Quote:
Originally Posted by TheFluff View Post
If you're asking for the why, distributing a small (couple megabytes at most) patch is a lot easier than redistributing the entire file.

If you're asking for the how, see this handy guide: http://www.srsfkn.biz/2008/11/11/mak...-with-xdelta3/
(xdelta3 is probably the best program for this; it works on both windows and unix, is easy to use and the executable size is small. You can of course use mkvmerge instead but it's a bit more annoying.)
Also, it gives guarantees as to what the resulting file will be like, unlike merging in a new script which will result in a different file for everyone (there's only an infinitesimally small chance of a similar file being produced with mkvmerge).
TGEN is offline   Reply With Quote
Old 2009-01-07, 17:37   Link #6
False Dawn
Florsheim Monster
*Fansubber
 
 
Join Date: Aug 2006
Location: UK
Age: 29
Quote:
Originally Posted by TGEN View Post
(there's only an infinitesimally small chance of a similar file being produced with mkvmerge).


Oh? Why's that? I always assumed that if I mkvmerge the same components into an mkv as someone else on a different computer, then we'd get the same results. Why isn't this the case? Just differences in settings, things like that?
False Dawn is offline   Reply With Quote
Old 2009-01-07, 17:48   Link #7
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 27
Quote:
Originally Posted by False Dawn View Post
Oh? Why's that? I always assumed that if I mkvmerge the same components into an mkv as someone else on a different computer, then we'd get the same results. Why isn't this the case? Just differences in settings, things like that?
Because there are things (like various unique identifiers (segment UID's, track UID's, attachment UID's etc etc) and other stuff) in the file structure that are chosen pseudorandomly upon muxing. The version of the muxing application is also stored, and IIRC the time/date of the muxing start is stored too. Which means the CRC will change every time you remux.

Of course, given the same version of the muxing application and control of the muxing settings (via a bat file), the file should always be functionally identical, even if the CRC isn't the same.


edit: obviously this works best for softsubbed stuff. If you need to change something hardsubbed, you should not reencode the entire file because that will change the coding in the entire file and the patch will have to diff the entire video, so it'll be huge. Instead just reencode the scene where the offending hardsubbed stuff is and do some cutting and splicing.
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Old 2009-01-07, 18:04   Link #8
TGEN
Far out, man!
*Fansubber
 
Join Date: Jul 2008
Age: 30
Quote:
Originally Posted by TheFluff View Post
edit: obviously this works best for softsubbed stuff. If you need to change something hardsubbed, you should not reencode the entire file because that will change the coding in the entire file and the patch will have to diff the entire video, so it'll be huge. Instead just reencode the scene where the offending hardsubbed stuff is and do some cutting and splicing.
Wildly off-topic, but I wonder what that would do to forward references just before the splice and backward references just after. Would the cutting/splicing tool account for that properly?

Last edited by TGEN; 2009-01-07 at 18:05. Reason: No need to quote the whole lot...
TGEN is offline   Reply With Quote
Old 2009-01-07, 20:09   Link #9
sneaker
Senior Member
 
Join Date: Dec 2008
Ureshii uses mkvmerge to patch the first episode of Kurozuka and the resulting file is 100% bit identical on all PCs.

Here's the batch they're using (they provide a binary of mkvmerge and the corrected .ass file with it):
Code:
@echo off
SET PATH=""
:start
echo Patching [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv....
if not exist [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv goto notfound
>NUL "bin\mkvmerge" -o "[Ureshii]_Kurozuka_-_01v2_[AAC-H264-720p][F76DF54F].mkv"  --language 1:jpn --track-name "1:Kurozuka 01: Adachigahara" --default-track 1:yes --display-dimensions 1:1280x720 --language 2:jpn --track-name "2:2.0 LC-AAC" --default-track 2:yes -a 2 -d 1 -S "[Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv" --language 0:eng --track-name "0:Styled ASS" --default-track 0:yes -s 0 -D -A "kurozuka01[FINAL]v4.ass" --track-order 0:1,0:2,1:0 --title "Kurozuka 01: Adachigahara" --engage no_variable_data
goto end
:notfound
echo -----
echo FILE NOT FOUND!!
echo Please copy your "[Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv" with the 
echo ORIGINAL FILENAME to this folder an run this patch...
pause
goto start
:end
echo DONE!
echo Deleting v1 file...
echo -----
del /f /p [Ureshii]_Kurozuka_-_01_[AAC-H264-720p][48F46569].mkv
They even put the correct crc32 of the output file into the output filename and they're using ordered chapters.
sneaker is offline   Reply With Quote
Old 2009-01-07, 20:15   Link #10
martino
makes no files now
 
 
Join Date: May 2006
Quote:
Originally Posted by TheFluff View Post
The version of the muxing application is also stored, and IIRC the time/date of the muxing start is stored too. Which means the CRC will change every time you remux.
Code:
--engage no_variable_data
will not store the time of muxing (as well as any other data that can change from one mux to another) and thus produce the same CRC if muxed with the same version (as you mentioned).

EDIT: beaten by sneaker... kinda.

I was always wondering though whether the same mkvmerge version across different platforms would produce the same CRC... lazy~

Anyway, I'd just like to add that in the past patching through a bat file and mkvmerge used to be more common than xdelta3. Seems to be the other way round these days... actually, I don't remember seeing an mkvmerge patch in a while now.

Personally I think it's for the better, cross-platform patching is more reasonable this way (or at least I think; only if we could see some bash scripts along those batch ones, not like it's much copy-paste work for the end user in the end so just a matter of effort), although I've had some weird problems with an xdelta3 patch made under win32 and applied under *nix. Who knows what that was...

Still, the amount of patches around and v2s is in a rather sad ratio (mainly when softsubbed MKVs are concerned). It sure would be nice to see more of them around in the future. Even though, who is to complain with BT around?
__________________
"Light and shadow don't battle each other, because they're two sides of the same coin"
martino is offline   Reply With Quote
Old 2009-01-07, 21:29   Link #11
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 27
Quote:
Originally Posted by TGEN View Post
Wildly off-topic, but I wonder what that would do to forward references just before the splice and backward references just after. Would the cutting/splicing tool account for that properly?
depends on what cutting/splicing tool you use, if you use mkvmerge you can only cut at IDR-frames so the problem solves itself.

I had forgotten about --engage no_variable_data, but xdelta3 also has the added advantage of being a lot easier to use for unix users that can't use the .bat file. Then again you could just provide a shell script too.
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Old 2009-01-08, 11:33   Link #12
TGEN
Far out, man!
*Fansubber
 
Join Date: Jul 2008
Age: 30
Quote:
Originally Posted by martino View Post
only if we could see some bash scripts along those batch ones
Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!
TGEN is offline   Reply With Quote
Old 2009-01-08, 13:18   Link #13
edogawaconan
LOL?
*Fansubber
 
 
Join Date: Aug 2006
Location: Indonesia
Send a message via MSN to edogawaconan Send a message via Yahoo to edogawaconan
Quote:
Originally Posted by TGEN View Post
Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!
is it even possible to make sanely sized & statically linked mkvmerge?
Irrelevant if using xdelta
__________________
edogawaconan is offline   Reply With Quote
Old 2009-01-08, 14:16   Link #14
TGEN
Far out, man!
*Fansubber
 
Join Date: Jul 2008
Age: 30
Quote:
Originally Posted by edogawaconan View Post
is it even possible to make sanely sized & statically linked mkvmerge?
Irrelevant if using xdelta
Distributing binaries along with it is a bad idea, as not everybody is running Obesux.
Code:
MKVMERGE=`env mkvmerge`
if test "x$MKVMERGE" = "x"; then
        echo "go cowpile mkvmerge for your platform"
else
        $MKVMERGE foo bar baz quux etc
fi
TGEN is offline   Reply With Quote
Old 2009-01-08, 19:10   Link #15
checkers
Part 8
*IT Support
 
 
Join Date: Jul 2006
Location: Western Australia
Age: 26
Send a message via MSN to checkers
Quote:
Originally Posted by TGEN View Post
Except don't make it a bash script; bash isn't 100% POSIX compliant. Write portable Bourne scripts, please!
Fansubbers don't care about Mac compatibility, why would they care about NetBSD 1 compatibility?
checkers is offline   Reply With Quote
Old 2009-01-09, 00:04   Link #16
Starks
I see what you did there!
*Scanlator
 
 
Join Date: Apr 2004
Age: 26
Send a message via AIM to Starks
Quote:
Originally Posted by TheFluff View Post
depends on what cutting/splicing tool you use, if you use mkvmerge you can only cut at IDR-frames so the problem solves itself.

I had forgotten about --engage no_variable_data, but xdelta3 also has the added advantage of being a lot easier to use for unix users that can't use the .bat file. Then again you could just provide a shell script too.
xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>
__________________
Starks is offline   Reply With Quote
Old 2009-01-09, 02:04   Link #17
edogawaconan
LOL?
*Fansubber
 
 
Join Date: Aug 2006
Location: Indonesia
Send a message via MSN to edogawaconan Send a message via Yahoo to edogawaconan
Quote:
Originally Posted by Starks View Post
xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>
> cd /usr/ports/misc/xdelta3/ && sudo make install clean

> sudo apt-get install xdelta3

(I don't know how to install it but it exist on Arch)

etc
__________________
edogawaconan is offline   Reply With Quote
Old 2009-01-09, 03:52   Link #18
TGEN
Far out, man!
*Fansubber
 
Join Date: Jul 2008
Age: 30
Quote:
Originally Posted by checkers View Post
Fansubbers don't care about Mac compatibility, why would they care about NetBSD 1 compatibility?
Hey, the oldest NetBSD I have running is a 1.4.2 (VAX even!), no need to exaggerate...
TGEN is offline   Reply With Quote
Old 2009-01-09, 04:30   Link #19
Nicholi
King of Hosers
 
Join Date: Dec 2005
Age: 31
/me approves of xdelta3 patching

It is certainly easier to compile than mkvtoolnix at least, lol. Could be packaged with a script as well for easy newb use. Also would allow you to fix more than JUST script errors, since you would be able to make a binary diff for literally anything (even re-encoded video segments). Forgotten fonts, chapters, tracks, other attachments, anything really.
Nicholi is offline   Reply With Quote
Old 2009-01-09, 12:32   Link #20
TheFluff
Excessively jovial fellow
 
 
Join Date: Dec 2005
Location: ISDB-T
Age: 27
Quote:
Originally Posted by Starks View Post
xdelta works on linux without wine? if not, then compiling xdelta from source doesn't sound easier. </linux user>
as edogawaconan pointed out, it's a unix program that has been ported to windows, not the other way around
__________________
| ffmpegsource
17:43:13 <~deculture> Also, TheFluff, you are so fucking slowpoke.jpg that people think we dropped the DVD's.
17:43:16 <~deculture> nice job, fag!

01:04:41 < Plorkyeran> it was annoying to typeset so it should be annoying to read
TheFluff is offline   Reply With Quote
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -5. The time now is 08:27.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
We use Silk.