Recently I've been asked once again to give some hints how to deal with dual-release encoding issues. Since I've got some spare time, I've compiled this little HOWTO describing an optimized workflow for encoding dual releases, in the hope that one or two encoders might find it useful. Constructive feedback and improvement suggestions are always welcome.
Synopsis
This little text describes the workflow I'm currently using to encode dual version anime releases - one high resolution softsubbed MKV release and one standard resolution hardsubbed AVI release, from a 120fps raw source. For the sake of simplicity, I'll skip over variable framerame encoding, which gets VERY complicated in the dual-release context, and assume normal constant framerate sources - if there is serious interest in this, I might write another article later.
It is by no means the "only" way to do it, and even less the "best" way, but it's a workflow which has been developed and refined over the years and has proven its worth. One can argue that there are newer tools available for the tasks, but I prefer to stick with the "tried-and-true" versions until some new feature surfaces which really justifies an upgrade.
Requirements
Except for sufficient knowledge in encoding theory and handling the basic tools (virtualdub, x264/MeGUI etc.) you need at least basic knowledge in avisynth scripting, and you need sufficient free diskspace on your encoding machine. Minimum is 15GB for normal-resolution sources and 30 GB for high-resolution sources. The tools I'm using are:
Virtualdub
Virtualdubmod
Avisynth 2.5.6a (you can try 2.5.7 if you want, I prefer the older version)
x264 with MeGUI
MKVtoolnix
Aegisub
Audio tools (Besweet, NeroAACenc, etc)
Workflow
1) Raw selection
Usually, most groups will have access to japanese P2P networks like Winny or Share and thus have access to various types of raws. In general, the translator can start with the first raw available, but nothing affects the quality of a release more than the raw selection. The best encoder can't turn a dogpile into caviar, so pick your release raw with care. If you happen to work in a group which leaves this decision to the encoder, just the better.
2) Loading the Video
It's very common nowadays to get raws encoded in 120fps, which is AVI's kludge to incorporate pseudo-VFR (23.976 and 29.97fps sequences) in one file. Since this is impossible to play back properly after manipulation, we need to decimate this video down to the target constant frame rate (usually 23.976, sometimes 29.97) due to the AVI container limitations.
There are many different ways to do this decimation, and the process is non-deterministic: The different approaches lead to frames being off by 1(!). This is very critical for scenetiming and especially for typesetting, so you should
prepare a workraw in any case. The idea is to make sure that timers and typesetters always use this easily-handleable workraw and NOT on the clunky original raw.
This is an example how I decimate the video for Hayate via avisynth script:
Quote:
|
directshowsource(fps=29.97,convertfps=true,"Hayate 30.avi").converttoyv12()
|
This decimates the framerate to 29.97fps, and it also guarantees YV12 colorspace for later. I'm usually careful and crop 2-4 pixels away from each side of the video, and then resize back up to the target resolution, but that's up to everyone to decide.
However, if you plan to release a h264 version, make sure you resize to mod16 - that's usually 704x400, 640x480, 1024x576, 1280x720. If you're strapped for diskspace, do the Audio step 3) first now and then disable the audio in Virtualdub. Otherwise just proceed, select Xvid single-pass with quant 3, and
disable b-frames(!)
The point is that the workraw (TSraw) should have no b-frames or anything which would make the job more error-prone for the timer and typesetter. Quant3 makes for a decent, albeit not amazing video - it should be close enough to the final result to make sure that the typesetter can work with fine details too. On the other hand it shouldn't be TOO large too, since it needs to be downloaded. If your timer is on a terrible line in Portugal, you can also create a "Crapraw" with quant 10 - easily downloaded, and it still has the keyframes he'll need for the Aegisub timing postprocessing later.
Always work with the FULL resolution. Even if the avi will be smaller eventually, the typesetter should work on the full resolution instead. The AVI will only be downscaled eventually. This way, you only work on ONE version, not on multiple ones.
Once the TSraw is encoded, do the Audio step 3) and mux the created MP3 audio into the TSraw for the final version. Give it an easily understandable name (e.g. Hayate-30-TSraw-xvid-nobframes.avi), upload it to your project FTP server and notify the others that it's up.
3) Audio extraction and preparation
There are several possibilities here. If your release contains audio in a form which you want to reuse "directly" without re-encoding it, you can just extract it via Virtualdub (using the _original_ raw, not the directshow line quoted up above, because it will force a conversion to WAV). In most cases however, the bitrate will be too high (192 or up) to fit your target releases, or you will want to reencode in Vorbis and AAC in lower bitrates to save space. Here, I tend to use VirtualDub to create a WAV file called "Audio.wav" of ~270 megs. Then, use your favorite audio tools to convert it to an MP3 for the AVIs and either a vorbis or AAC version for your MKV. If you want to keep the original audio of a MKV or MP4 source for your own MKV release, you don't need to do anything - just load the original raw in mmg later and only tick the audio to be muxed in.
4) Filtering of the original raw
Filtering of raws, especially high-resolution raws, can be very very time-consuming (depending on the encoder's computer horsepower available, we're talking several hours for a pass). Therefore, it's very efficient make only a single filterpass, and to store the result in a lossless AVI.
So, we create another avisynth script, with EXACTLY THE SAME (!) decimation and cropping/resizing lines, but this time with the desired filtering too. I prefer Lagarith as lossless codec, but ffv1 is fine too, or huffyuv (though the latter is fairly inefficient and takes more diskspace).
Save the file to an equivalent of "Hayate-30-prefiltered.avi" and go to bed. This will usually take a while. Time for the others (translator, editor, timer, typesetter etc) to do their job. You sleep and pick up their work results in the morning.
5) Adding AFX signs
If you don't use AFX, just skip this point. If you do, then add them to the _prefiltered_ lossless AVI. The result should be an avisynth file which loads the prefiltered AVI and overlays all AFX signs. Make sure you check each sign visually, for bleeds or other glitches.
6) Adding ASS signs
Time to open up Aegisub and to either load the Step 5)-generated Avisynth script or the prefiltered AVI directly. Then, load the ASS signs script from your typesetter (always insist on a seperate file, NEVER add signs to the main dialogue script!)
Go through all entries and check all ASS signs, to make sure that they're clean. It's also a good idea to make a "Fonts collector" check in Aegisub to make sure that you have all fonts installed.
The result should be a checked and clean ASS script.
7) Shifting/Checking the Karaoke
If you're lucky that others shift the OP/ED/Insert Karaokes for you, only make quick xvid one-pass ranged OP/ED test encodes in Virtualdub, with the Audio.wav extracted in step 3) selected as "WAV Audio". If all is good, proceed. If not, fix the timing or poke whoever does this in your group.
8) Putting it all together
Now it's time to create one more "final" lossless video. It should contain everything but the dialogue script. Since you have checked all signs and karaokes already, the only glitches which should appear from now on are in the dialogue, so it's smart to finalize anything else.
Load the AFX-adding avisynth file in vdub, and add all Karaoke and ASS sign scripts. Then, pick your favorite lossless codec and encode everything to the equivalent of "Hayate-30-final.avi". Take a deep breather. Most of your pain is over now, and any reencodes required later than this point will be VERY fast.
9) Encoding the MKV and preparing the Softsub MKV
Assuming that you're releasing the MKV as softsub, you can _immediately_ proceed to encode the final version. Just stick the avs wrapper script for the -final.avi into MeGUI and encode away. You do NOT need to wait for any finalized scripts. Very convenient.
If you want to save time, you can also start to write the chapters file and prepare mmg for the final muxing, tagging all streams, adding the fonts etc.
10) Adding the dialogue script
So you've finally received the -RC or even -F dialogue script, depending on your group's setup. For the MKV, just make a ASS-SRT conversion if you do that, and add both the .ass and .srt script to mmg. Mux. Congrats, you just finished the first release candidate.
Before you upload it to your project FTP, I recommend that you open it in your player, check that all streams are properly tagged, and skip through the chapter markers to see that they all turn out okay. If they do, time to open the FTP client and push up the RC to the project dir.
Time to work on the AVI: Make a small avisynth wrapper which downscales the hi-res raw to the AVI resolution, load it in Virtualdub and add the dialogue script (or add it to the avisynth wrapper in the first place). Then, make a normal xvid 2-pass encode for the desired target size.
Finally, it's time to sit/lie down and proofwatch the entire MKV episode. If you happen to come across an error, fix it in the .ass (and .srt) scripts, remux the MKV and reupload the file. For the AVI, you only need to repeat the SECOND pass with the fixed .ass script. Usually the MKV upload takes longer than the second pass of the AVI, so you don't lose any time.
When the MKV seems to be all right, mux the now-finished AVI with the prepared audio in Virtualdubmod, save it to the target filename and upload it to the project FTP. It's a good idea to rewatch the episode in AVI once more, to make sure that no glitches popped up. Again, when errors show, just stop any upload, fix the error, remux the MKV, upload the MKV and reencode the second pass of the AVI. Since we're only working on lossless pre-encoded video, this can be done extremely quickly.
11) The RCs pass QC and are ready for release
Calculate the CRCs, add them to the RCs on the project FTP and notify the distro.
Done.