Creating Fansubs - Questions and Answers

[Jimmy C]

So, no matter what, this is impossible without a reencode of all videos to be combined?

[TheFluff]

Originally Posted by Jimmy C

So, no matter what, this is impossible without a reencode of all videos to be combined?

No, reread my previous post. If the two videos are encoded with different settings, you can either reencode both or you can find out (using one of several easily available analysis tools) what settings one of them was encoded with, and reencode the other to match the first one's settings.

[Jimmy C]

Could you name me one or two that would be most helpful in this case, please?

[checkers]

open the video with a hex editor (one is built into virtualdub). Visible within the first 30s of scrolling or so will be a string starting with "x264" that lists the encoding options.

[Scab]

find "x264" "c:\your.file"

[TheFluff]

I've always used the hex editor version or strings(1), but IIRC avinaptic and maybe mediainfo can do it too in a more user-friendly way.

[Jimmy C]

Originally Posted by checkers

open the video with a hex editor (one is built into virtualdub). Visible within the first 30s of scrolling or so will be a string starting with "x264" that lists the encoding options.

I got Hex Workshop to take a look at the mkv, this is from the mkv that I want to match my encodes to, is this what you want me to look at?

Spoiler for intended target:

x264 - core 65 r1057M 5f8a149 - H.264/MPEG-4 AVC codec - Copyleft 2003-2008 - http://www.videolan.org/x264.html - options: cabac=1 ref=5 deblock=1:1:1 analyse=0x3:0x113 me=esa subme=7 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=2 deadzone=21,11 chroma_qp_offset=-2 threads=1 nr=0 decimate=0 mbaff=0 bframes=1 b_pyramid=0 b_adapt=1 b_bias=0 direct=3 wpredb=1 keyint=250 keyint_min=25 scenecut=40 rc=2pass bitrate=1813 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

This is from one of my own encodes

Spoiler for my encode:

x264 - core 66 r1093M 1df50b9 - H.264/MPEG-4 AVC codec - Copyleft 2003-2008 - http://www.videolan.org/x264.html - options: cabac=1 ref=4 deblock=1:-1:-1 analyse=0x3:0x113 me=umh subme=8 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=-2 threads=3 nr=0 decimate=1 mbaff=0 bframes=3 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 keyint=250 keyint_min=25 scenecut=40(pre) rc=2pass bitrate=2114 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 vbv_maxrate=50000 vbv_bufsize=50000 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00

By poking around the x264 configurator, I was able to change most of the settings that were different between the two. Except for these few:

Spoiler for differences:

cqm=2 threads=1 (I think this is Auto, so should be ok) decimate=0 b_adapt=1(adaptive b-frames?) scenecut=40 (mine has "(pre)" after the 40)

mine has these two options that aren't in the target:
vbv_maxrate=50000 vbv_bufsize=50000
A second encode didn't have these lines.

But even after adjusting the settings, I still get the warning that the private data lengths is different at 37 and 170.
Am I even looking at the right place in the files for information? If not, how else can I find it?

[TheFluff]

Yes, those are the settings you're looking for. You're pretty much always going to get the warning though, the question is if it plays back correctly or not. You don't need to worry about the VBV settings, those are encoder-side only and do not affect decoding.

As for the other ones:
decimate=0 means --no-dct-decimate is enabled ("Disables coefficient thresholding on P-frames"). b_adapt is adaptive b-frames decision mode. You get "pre" after scenecut if you use more than one encoding thread, but it's an encoder-side only setting too so you don't need to worry about that either.

[Jimmy C]

Sadly 37 and 170 are too far apart to be viewable. Despite adjusting the settings, there was no change in the length of my encode.
What can you tell me about the settings I couldn't find? The ones in differences.

[TheFluff]

Originally Posted by Jimmy C

Sadly 37 and 170 are too far apart to be viewable. Despite adjusting the settings, there was no change in the length of my encode.
What can you tell me about the settings I couldn't find? The ones in differences.

Oh, I missed CQM. It was apparently encoded with a custom quantization matrix. Avinaptic can display what matrix was used; you need to use the same matrix with your encode.

[Dark Shikari]

If I recall correctly, the CQM is stored in the SPS, so if you just use --sps-id 1 in your encoding you should be fine for catting the two h264 streams together.

[Jimmy C]

I haven't gotten around to looking for the CQM with avinaptic yet. So, I just add "sps-id 1" to the custom command line?
Another thing, how do I input a custom quantization matrix into the x264 encoder? That option seems to be disabled there.

[checkers]

--cqmfile "C:\path\to\cqm.extensiondoesntmatter"

[TGBatman]

Hi guys. I've been lurking a lot over here, but today a question arised, so I decided to give it a try and register to ask it.

I've been wondering if tesa is really worth the lot of time it requires to encode. I haven't even tried exhaustive -me for like a year.

Also, I have some questions about megui and the min/max quantizers description. It says that these are disregarded when encoding in CRF or QP mode.

But it turns out that letting them at defaults (10/51) made x264 output a CRF 18 encode at 170 mbs, it looked really crappy. Then I've set max quantizer to 18, and now that made x264 output a 206 mbs file that looked much better.

Now, my question about it is: Is the description in MeGUI about these quantizers wrong, or I AM the one doing something really wrong?

My source is Shikabane Hime AKA DVD 01, my script (after IVTCing with AnimeIVTC) is:

Import("C:\yatta_6-129\PLUGINS\FastLineDarken.avs")

function Preset0(clip c) {
#Name: Filtering 1
c
dfttest(sigma=0.2).toon(0.2).addborders(2,2,2,2).awarpsharp(4,1,0.5,1).crop(2,2,-2,-2).gradfun2dbmod(thr=1.1,mode=2,chroma=false,str=0,temp=50,adapt=64,mask=true,radius=3,range=3,show=false)
crop(4,2,-4,-0).spline16resize(704,480)
return last
}

function Preset1(clip c) {
#Name: Filtering 2 (heavy grain parts)
c
toon(0.2).addborders(2,2,2,2).awarpsharp(4,1,0.5,1).crop(2,2,-2,-2)#.gradfun2dbmod(thr=1.1,mode=2,chroma=false,str=0,temp=50,adapt=64,mask=true,radius=3,range=3,show=false)
crop(4,2,-4,-0).spline16resize(704,480)
return last
}

DSS2("lossless.mp4")

PresetClip0=Preset0()
PresetClip1=Preset1()

PresetClip0.Trim(0,29338)+PresetClip1.Trim(29339,29369)+PresetClip0.Trim(29370,34065)

#Trim(2165,33691)
#trim(33692,34065)

I just copied Yatta layout to do the presets .

These are my x264 settings (copied from MeGUI's command window):

program --crf 18 --ref 16 --mixed-refs --no-fast-pskip --bframes 4 --b-adapt 2 --b-pyramid --deblock -1:-1 --subme 9 --trellis 2 --partitions all  --8x8dct --qpmax 18 --me umh --merange 32 --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input"

Would I be better using 3-pass VBV?

Help with improving settings to visibly improve output's image would be very appreciated

I don't know what else to tweak. Maybe adding more b-frames?

Bye

TGBatman

[Dark Shikari]

Originally Posted by TGBatman

Hi guys. I've been lurking a lot over here, but today a question arised, so I decided to give it a try and register to ask it.

I've been wondering if tesa is really worth the lot of time it requires to encode. I haven't even tried exhaustive -me for like a year.

No, it really isn't. It's on the list of "options to use if you absolutely don't care about time" (along with trellis 2).

Originally Posted by TGBatman

Also, I have some questions about megui and the min/max quantizers description. It says that these are disregarded when encoding in CRF or QP mode.

But it turns out that letting them at defaults (10/51) made x264 output a CRF 18 encode at 170 mbs, it looked really crappy. Then I've set max quantizer to 18, and now that made x264 output a 206 mbs file that looked much better.

You can't compare quality among files of different bitrates. Furthermore, you shouldn't mess with qpmin/qpmax. If you want to weaken AQ, do it by... directly weakening AQ, not arbitrarily crippling it with qpmax.

program --crf 18 --ref 16 --mixed-refs --no-fast-pskip --bframes 4 --b-adapt 2 --b-pyramid --deblock -1:-1 --subme 9 --trellis 2 --partitions all --8x8dct --qpmax 18 --me umh --merange 32 --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input"

You should raise deblocking on anime, not lower it. No-dct-decimate is also really not recommended unless you really need to keep dither or something.

Originally Posted by TGBatman

Would I be better using 3-pass VBV?

Why would you need VBV? And 3-pass is totally pointless.

[TGBatman]

Thanks for the suggestions Dark Shikari . Would 1:1 be a good deblocking setting? Or more would be safe without killing detail? That's my fear when using deblocking, detail killing.

And yeah, I want to keep the dithering caused by gradfun2dbmod.

Bye

TGBatman

[TheFluff]

Originally Posted by TGBatman

Would 1:1 be a good deblocking setting?

1:1 isn't extreme in any way, but as always, why don't you just try it for yourself

[TGBatman]

Yeah, that's what I did, and this morning I woke up to see a 158 mb video that looked gorgeous, and that with crf 18.

Again, thanks for the answers guys .

Bye

TGBatman

[guest0815]

I got h264 raws where in a few scenes the pixel are going rampant in dark flat areas. I wonder what is normally used on that kind of noise.

Spoiler:

The pics are cut 1:1 from a 720p raw:



I tried several filters but most of them didn't even touch those pixel when the rest of the image was already smoothed beyond recognition

fft3dfilter(sigma=2) smoothed it out a little bit but in a way it's making things worse because the noise that is fluctuating more or less equally spread over the dark areas gets clumped together making the fluctuations between frames more noticable (fft3d left a tiny little bit of temporal artefacting in there too btw)



What got vastly superiour results over everything else was SmoothD(1,4,0,4) but with those darkened areas around lines as a trade-off.

thanks

[checkers]

gradfun2db plus:

# Border processing fix for GrandFun by MugFunky and Alain2
# Edit by Soulhunter: Changed strength from int -> float 

Function GradFunkMirror( clip c, float "strength" )
{
	strength	=	default( strength, 1.2 )
	w			=	c.width()
	h			=	c.height()
	vflip		=	c.FlipVertical()
	hflip		=	c.FlipHorizontal()
	
	stackhorizontal( hflip.crop( w-16, 0, 16, h ).addborders( 0, 16, 0, 16 ),
	\	stackvertical( vflip.crop( 0, h-16, w, 16 ), c, vflip.crop( 0, 0, w, 16 ) ),
	\	hflip.crop( 0, 0, 16, h ).addborders( 0, 16, 0, 16 ) )
	gradfun2db( strength )
	crop( 16, 16, -16, -16 )

	Return last
}