Filters, Plugins, and the like

[Kurz]

I am learning how to clean up a source before I send it off to be encoded.
So this question is to the encoders.

What do you use to clean up your source, be it Plugins for AVIsynth, or anything else excetra...

P.S. It doesnt matter what the source is... be it TV capture or DVD.

[gumbaloom]

AVISynth filters.

However the filter(s) will depend on the source.

-gumbaloom

[TheFluff]

Personally I never use vdub filters in vdub (they can be loaded in avisynth anyway) - I only use avisynth. What specific filters I use and at what settings depends a LOT on the source, and there is no general answer, or "always use" filters (except maybe undot). But anyway, a few favorites of mine, that tend to get used quite often:
- temporalsoften (avisynth builtin)
- undot
- hqdn3d
- pixiedust
- msharpen
- deen
- fft3dfilter
- warpsharp (the vdub one mainly, or the sekrat japanese one)
- aa (a script, not a plugin)
- limitedsharpen (script)
- unhalo (script)

[Sylf]

What everone does are same thing:

1) remove noise from the flat area - increase the compressibility
2) kill the edge noise, such as edge enhance, halo/ring, rainbow, etc
3) sharpen a image of necessarily, so it doesn't look a total blur.
4) correct the color intensity, saturation, etc if needed

How those are done? That's the part really depends from person to person.

I think for a starter, some 2d and/or 3d softening filters like convolution3d, deen, mipsmooth are fun toys to play around with. Learning to use interleave() and/or subtract() to see what's changed with different filters/settings will be really revealing.

[RaistlinMajere]

Trade secrets

[Kurz]

Thanks TheFluff! I really appreciate that post. I was using MSharpen and Temporalsoften... however I wasnt getting the results I wanted.
(Yes I do mean AVIsynth)

Btw Whats with all these Trade Secret Posts? (I Noticed this from another Forum)
Are Some of the modern encoders that secretive about which tools they use?
Its not like I can magically become as experienced as others by getting these tools. I thought Experience was the Major factor of being an Encoder.

[RaistlinMajere]

It is, and you should start simple at first. Don't use stuff like dust until you're somewhat experienced.

[TheFluff]

Well, some encoders are like typesetters in that regard. Not all are, however, and there are places where you can ask for (and sometimes receive) advice. Darkhold is one, the doom9 forums another.

[Eeknay]

Just adding a few filters to the list that I like; FluxSmooth, VagueDenoiser, Unfilter (well, sometimes anyway), RemoveGrain, deblock_qed, dehalo_alpha. Also have a play around with Levels and Tweak.

And yes, #darkhold and doom9 are both excellent resources for learning.

[DryFire]

Originally Posted by TheFluff

- limitedsharpen (script)

I'd suggest limitedsharpenfaster (script + support dll)

I'd also like to add removegrain(mode=1) == to undot() but may be faster if you use the sse2 version and have a cpu the supports it.

Some more filters:
FreezeFrame (internal)
degrainmedian
gradfun2db (sometimes it can be interesting)

[Zero1]

I don't use filters, Chuck Norris is a personal friend of mine, so I just get him to stare at the video until it makes itself perfect. If it's so bad, I send Chuck round to the company in question to roundhouse kick their staff and go "borrow" the masters.

Doom9 is Chuck Norris certified, so go check it out, but remember to search before asking if you have a question, it's always possible it's been answered before.

The key is not to go crazy on sharpening filters, since those will cause haloing and make the source more difficult to compress, also take it easy with cleaning. Sometimes it can do a good job in some areas, but may totally make a mess of other areas.

Basically use everything in moderation and remember that filtering too much looks as bad as, if not worse than the original video.

If you've got spare time, grab a variety of sources and practice with them. In particular get something analog. Grain and rainbows are fun to deal with. Then maybe find some DVB rips, poor ones may have blocking, so give those a try. Others might be oversharpened by the raw capper, that will be good practice for halo removal.

[RaistlinMajere]

Dot crawl is even more fun

[Alizar]

Some filters I've had good non-image breaking experiences with:

-masktools (use it to mask my lines so as not to smooth them quite so much)
-removegrain (it has a built-in repair function to help catch its own artifacting)
-tbilateral (Careful, even on the defaults, this will eat some detail!)
-ttempsmooth (set the scene-thresholding right, and it's a beaut, also feeding it a pre-processed clip to use for scene/noise detection is great)
-fft3dgpu (used judiciously with a low sigma for postprocessing)
-vaguedenoiser (if it the video really sucks that much, though it probably doesn't)
-convolution3D (I don't this so often anymore, but it does good work also)
-mfrainbow (a script, you can find it on doom9, which gives good results when dealing with rainbowing)
-bifrost (temporal derainbowing, will work on stationary stuff, but on moving it's less than useful use mfrainbow for that)
-undot (it's a 3x3 median smooth filter and fast as heck. Why not?)
-blockbuster (used slightly turned down from the defaults this can tone down DCT-blocking in xvid. Note that for h264, there are more effective ways that are part of the Sharktooth and ChronoCross builds of x264)

I realize some of these have been mentioned before, of course.

[Access]

Also, minimize any colorspace conversions, esp. to/from RGB colorspace. Use the 'fast re-encode' or whatever-it's-called option in VDub to encode straight from the avisynth colorspace.

Like he says, hold off on too much sharpening, so much of the stuff I see out there is oversharpened / sharpening artifacts.

As far as secretive nature, it's b'cos if anyone can learn how to encode well, many encoders fear they would be 'out-of-a-job', or replaced in their own groups by someone else. It's not as bad as it used to be, most of the good filters have already been mentioned above. Or you can just follow the links from http://www.avisynth.org/warpenterprises/ and try them all for yourself. Even with these filters, it takes a lot of trial-and-error as you still need to set the parameters and those will often depend on the actual video source.

[TheFluff]

Originally Posted by Access

As far as secretive nature, it's b'cos if anyone can learn how to encode well, many encoders fear they would be 'out-of-a-job', or replaced in their own groups by someone else.

Not really, at least not for me. A rule of thumb says there are mainly two types of encoder:
1. The one who does it the "load AVI in vdub, press button, receive xvid" way, and
2. The one who actually knows what he's doing.
There's a minority of people in between, too, who seems to like putting lots of avisynth filters in random chains.

Anyone can attach subs, it doesn't take more than about 10 clicks in virtualdub. Actually encoding something takes some experience and a bit of effort. Encoding something good, or something with things like VFR, takes a fair amount of actual technical knowledge, a good bit of experience, and sometimes ridicolous amounts of effort (*coughayakashicough*).
In any case, the docs are already out there, anyone with a good working brain and some computer experience can learn to make decent encodes, especially if there's good raws. It's not even particularily hard. All it takes is some RTFM-ing and some experimenting with filter settings. Having a tutor helps, but isn't strictly necessary. The thing is that encoding is (or at least has been) one of the most under-appreciated jobs in the fansub community, and many people consider encoding boring, overly geeky or just not worth it.

I don't feel particularily threatened in my position as an encoder. Encoding isn't like typesetting, there aren't really any secret "tricks" or stuff that people will go "hey, that's cool" at. Most things encoders are proud of are stuff that only other encoders would notice... >_> However, I do think that the fansub community needs more good encoders, not less, which is why I gladly share what knowledge I have when people ask.

[RaistlinMajere]

AVISynth stuff is a PITA to teach to newbs though

[Quarkboy]

Maybe I'm an exception, but I learned all this stuff by a combination of

1. Trying various things.
2. Reading readme's.
3. Reading doom9
4. Reading threads like these on animesuki.
5. Asking elders for advice on occaision.

I never really had anyone "teach" me any of the basics. Just read the documentation, experiment, form your own opinions, read other people's opinions, rectify the two, and you're done.

Filters I've used are fastlinedarken, ttempsmooth, fft3d, tfm, tdecimate, tdeint, msharpen, deen, awarpsharp, fluxsmooth, manaodenoise, bifrost, blendbob, and, um... undot. But not all at once .

For nice DVD sources I use a small fluxsmooth and a -nr 50 on x264, and that's it (besides IVTCing issues). For a film grain filled mess I use manaodenoise, deen, ttempsmooth, awarpsharp, msharpen, fastlinedarken, approximately, in that order. Oh, and I've modified manaodenoise a bit .

[Mentar]

There was a time when encoders had this strange notion to hide their secrets and protect their scripts, especially 5 years ago. But times have changed quite a bit - I believe that #darkhold was an important point of change. Today, if you have any specific problem you can describe in a proper way (for example via screenshot and clips), you can go there and (politely) ask people for help. Unless you've come to a very bad time, you'll usually get help and offered options to deal with the issue you mentioned. Especially in the DVD world there's alot of cooperation and tech exchange between encoders, which is one of the main reason for the quality surge in the recent years.

In the fansub world, there is more inertia to overcome, but since recently more DVD encoders are seeping into the ranks of fansub encoders, it's slowly changing for the better, too.

[Alizar]

Originally Posted by TheFluff

I don't feel particularily threatened in my position as an encoder. Encoding isn't like typesetting, there aren't really any secret "tricks" or stuff that people will go "hey, that's cool" at. Most things encoders are proud of are stuff that only other encoders would notice... >_> However, I do think that the fansub community needs more good encoders, not less, which is why I gladly share what knowledge I have when people ask.

I agree with this whole-heartedly. And although I don't hold my work up a a paragon of 'omfg-awesomeness', I'll gladly share any experience I've gained (even filter chains that've done well by me) if any one cares to ask in such a format as I can reasonably help (i.e. talking on IRC = good, trying to troubleshooting encoding woes over email = bad). And of course, I'll happily second all of the sources for knowledge mentioned already in this thread.

I figure if we all share that means all the fewer encodes where I feel the need to wince when I open them. And that's a win in my book.

Edit: If I can be replaced by someone using my filtering chains, I'm not paying my rent as an encoder.

[TheFluff]

Originally Posted by Quarkboy

Maybe I'm an exception, but I learned all this stuff by a combination of

1. Trying various things.
2. Reading readme's.
3. Reading doom9
4. Reading threads like these on animesuki.
5. Asking elders for advice on occaision.

I wouldn't say you're an exception, I learned in a similiar way myself, and I know more than a few encoders that did the same. You can't really teach someone insanity ;P