Encoding documentation for total newbies

[Hinoe]

Hello,

I am a Brazilian fansubber, specialized in editing, and I am willing to learn to encode.

That's enough introduction; now for the facts that outline the issue.

I have been reading some Avisynth documentation (at its wiki), but it's not helping me a lot. It's generally quite overwhelming when present, or else absent/incomplete. The biggest issue is that there seem to be hundreds of filters for each thing I may want to do and there's no telling which way I should go. It's probably safe to say it's useless to me, although it will most certainly help more experienced people.

I'm also not that much knowledgeable about technical stuff like banding, grain, artifacts, and other video problems (or audio problems for that matter); I know when something looks/sounds horrible, but I can't tell what the problem is from this technical POV. I tried to read stuff up in Wikipedia but soon gave up.

As if that wasn't enough trouble, the x264 settings just confuse me. Too many settings, too little explanation of what they do and how they affect stuff. Doing some search I found this and this, I'll be trying those out soon.

By now you see what the problem is. I have trouble with finding documentation that will give me the information I need without making me want to bang head against wall because I don't have a clue of what it's telling me. I also need the documentation I read not to be too outdated, otherwise I could just go read about encoding in DivX3.

So, the request: can you please point me to good newbie guides on the many aspects of encoding that won't confuse me too hard?

Also, a question: perhaps I would need someone to guide me? Not that there is anyone with the knowledge required and the time to properly do that...

Thanks in advance.

EDIT: Perhaps it would be interesting if the experienced people (/me looks @ TheFluff) shared a bit on their path to encoding knowledge. If nothing else, at least I'll know the learning curve is indeed as steep as it has been seeming to be and I'm not the first to have issues with it.

[cyth]

Originally Posted by Hinoe

I have trouble with finding documentation that will give me the information I need without making me want to bang head against wall because I don't have a clue of what it's telling me.

This is the reason why so many encoders end up crippled or otherwise impaired, because they have to go through that process.

[TheFluff]

The problem isn't really that you're dumb or that the tutorials are bad, it's that the concepts involved simply aren't that easy to understand. Interlaced video may seem easy at a glance but the implications are wide-ranging and telecine has never been a trivial process. I can try to (and frequently have) explain things in more newbie-friendly ways, but I cannot make the underlying concepts easier to understand. The fact that "encoding" covers a very large range of distinct topics doesn't really help, either.

The "traditional" way of learning these things is the one you're using: banging your head against a wall for months at an end. I did it mostly the same way, by hanging around people who seemed to know what they were doing and listening to what they were saying despite not understanding much of it, printing out various documentation and reading it on the bus, experimenting with things on my own and failing again and again and again. I can't really offer any help with teaching methods; the way I learned is a pretty good way to lose interest in encoding.

Most of the above applies to theoretical concepts; learning filtering is in large part a matter of experimentation, especially since filtering is subjective. I was never very good at it (mostly due to a lack of patience) and what I know is mostly outdated now anyway. I prefer not to filter very much, anyway.

x264 options aren't that interesting either; sure it's nice to be able to tweak a few options and it's useful to understand how the encoder works, but it's not essential. Especially not since you should be using the presets/tunings, which were put there by people who most definitely know what they're doing.

In the end, learning to encode is largely a matter of learning by doing; I'm not knowledgeable because I've spent a lot of time studying this, I'm knowledgeable because I've been doing stuff related to it, talking about it, explaining it and experimenting with it for years on end. I think the only recommendation I can give is to find someone that will put up with stupid questions, ask a lot, read a lot and do a lot of trial and error things, and you'll eventually pick things up.

edit: also, it's important to understand what you're doing. Don't just copypaste filter chains or x264 settings just because, always try to understand why they look like they do.

[Desbreko]

I learned everything I know about encoding through hours of reading information from wikis, forums and other documentation, and even more hours of experimentation. So about the only thing I can say is, "Keep at it."

[KoutetsuAnes]

I think you could use this website for information about some filters and their options.

[Hinoe]

Hello,

I guess although it is sad that the learning curve is indeed as steep as it seems, it is also good to know that everyone else has been through this.

I hope you guys can put up with my noob questions, then . I, for one, were always one of the first to go and tell people to RTFM; it's the first time that I'm actually not helped by The F***ing Manual (is swearing allowed here? The rules aren't very specific about it, unless I'm blind) as much as I'd like... This is actually interesting, it tickles my knowledge thirst; I'm going to keep at it.

Other than that, thanks for the support, and thanks KoutetsuAnes for the link, I'll check it out.

[jfs]

I'm thinking that perhaps one thing that's lacking might be a general overview, a bit in depth but not really detailed. Maybe it already exists.

[Hinoe]

jfs,

What you have conveyed in so few words is my issue (and apparently the issue of many others) exactly. To me, such general overview, if one was ever made, should point out things like "what [banding/halo/artifacts/grain/blur/blah/...] is, how to detect it, what filters are available to fix it, and what you should be careful with when using them" (aka "hey, don't warpsharp to death"), or "what x264 settings you should REALLY avoid and why" (hello huge lookahead, I hear you cause blocking in anime + crf). It doesn't need to explain every single detail about the filters/settings, just give a general overview, and perhaps a few links to more details for the search-lazy.

From that point on, it would probably be much easier to tackle stuff like filter documentation, because what you read suddenly isn't complete random garbage anymore.

[Dark Shikari]

Originally Posted by Hinoe

"what x264 settings you should REALLY avoid and why"

"Anything other than CRF, preset, and tune". See, that was easy.

Originally Posted by Hinoe

(hello huge lookahead, I hear you cause blocking in anime + crf).

Erm, what?

[KoutetsuAnes]

You could check this forum too it has good info.
You could also lurk in #encoders @ irc.rizon.net even though it's mostly empty or just lurk in irc channels until you get to know some encoders.

[Hinoe]

Originally Posted by Dark Shikari

"Anything other than CRF, preset, and tune". See, that was easy.

lol.

Come on, I'm serious. There are so many changeable settings for a reason, right? ......right?

Originally Posted by Dark Shikari

Erm, what?

Hmm... This is going to get somewhat off-topic, but...

There's this encoder I know (considered one of the greatest anime encoders of Brazil afaik). He said that he noticed that if he used "for example, x264 --crf 18 -o video.mkv script.avs, the image quality is actually worse than x264 --crf 18 --no-mbtree -o video.mkv script.avs", due to blocking appearing in the non -no-mbtree video. He mentioned this issue was more noticeable when working with anime and crf, but didn't say what other ways of producing it there were. He's using x264 build 1659 ("yeah that's right I haven't updated it to 1666"), but says he noticed this since the introduction of lookahead. He also added that the issue has been lessened heavily since the first days of lookahead, and that it was way worse at the beginning.

-- UPDATE --

He just gave me some screenshots. I should tell you I DON'T SEE ANY DIFFERENCE HERE, and then again the resolution is a joke (640x360 for HD source lol), but he claims he does.
http://img341.imageshack.us/img341/4877/mbtree.png -> with lookahead
http://img192.imageshack.us/img192/4519/nombtree.png -> --no-mbtree
http://img266.imageshack.us/img266/193/lossless0001.png -> source (BD -> avs -> screen taken in vdub) used for the encodes
Also, that was crf=20.

-- UPDATE FOR SOME MORE SCREENS --

Now for some crf=18 720p:
x264 options and output: http://pastebin.com/MxVMppCW
with mbtree: http://img822.imageshack.us/img822/5807/commbtree.png
without mbtree: http://img708.imageshack.us/img708/6870/semmbtree.png
"This setting also generates a horrible amount of banding".

-- I SHOULD SAY, FOR GREAT JUSTICE, THAT: --

I don't see any banding OR blocking... Either he's a visual genius, or I'm blind. Also, his friend said the with mbtree one has "quadriculado", layman's term for blocking. Dunno what to make of this now. (that my laptop screen sucks. Hello, HP.)

[Dark Shikari]

Originally Posted by Hinoe

lol.

Come on, I'm serious. There are so many changeable settings for a reason, right? ......right?

Yes, for the developers to test with. Don't touch anything else, or I'll do what Theora did and remove them permanently so that people will stop messing with them.

Originally Posted by Hinoe

There's this encoder I know (considered one of the greatest anime encoders of Brazil afaik). He said that he noticed that if he used "for example, x264 --crf 18 -o video.mkv script.avs, the image quality is actually worse than x264 --crf 18 --no-mbtree -o video.mkv script.avs", due to blocking appearing in the non -no-mbtree video.

Two videos of different filesizes have different quality? You don't say!

Almost anyone who describes themselves as an "encoder" is a clueless idiot who does nothing but poke at random buttons all day in the deluded belief that it helps them produce better results. Almost without fail, they know absolutely nothing about the internals of the options that they are setting. Don't be an "encoder".

[Hinoe]

Originally Posted by Dark Shikari

Yes, for the developers to test with. Don't touch anything else, or I'll do what Theora did and remove them permanently so that people will stop messing with them.

Point taken.

Originally Posted by Dark Shikari

Two videos of different filesizes have different quality? You don't say!

I'm not much worried about filesize, I'm more worried about his claim that lookahead induces blocking (and apparently now also banding). Also, the way you're putting it makes it seem like every compressing option will always fuck up quality for good. That's the same as saying "lossless PNG is smaller than BMP / FLAC is smaller than WAV, therefore it is of worse quality". And I still do not know whether lookahead does induce blocking or not.

Originally Posted by Dark Shikari

Almost anyone who describes themselves as an "encoder" is a clueless idiot who does nothing but poke at random buttons all day in the deluded belief that it helps them produce better results. Almost without fail, they know absolutely nothing about the internals of the options that they are setting. Don't be an "encoder".

Now, now, there's no need for all that rage. He made a claim, I'm not sure whether it's true or not, it's been put for analysis, that's all there is too it.

[Dark Shikari]

Originally Posted by Hinoe

Point taken.

I'm not much worried about filesize, I'm more worried about his claim that lookahead induces blocking (and apparently now also banding). Also, the way you're putting it makes it seem like every compressing option will always fuck up quality for good.

No, what I'm saying is you shouldn't touch things you don't understand. The defaults are there for a reason. Pressing every button available to you doesn't make you smarter and doesn't get you any better results. It might make you feel better though, especially if you wasted days doing it.

Furthermore, you've already committed the cardinal sin of encoding by comparing two videos encoded at different bitrates and making a judgement about quality.

[Kristen]

You also tend to learn quite a lot through practice and trial/error. Start with something minimalist, like x264 settings somebody else tells you to use and maybe a few filters, try an encode, and see if you like it.

[Meltingice]

Originally Posted by cyth

This is the reason why so many encoders end up crippled or otherwise impaired, because they have to go through that process.

Originally Posted by Desbreko

I learned everything I know about encoding through hours of reading information from wikis, forums and other documentation, and even more hours of experimentation. So about the only thing I can say is, "Keep at it."

Easy way out is to befriend a good encoder and then constantly harass him/her with questions.

[cyth]

Your friends must be pretty lucky to have you.

[Quarkboy]

Being a good encoder is like being a good scientist.

It's not what you know, but how good you are at knowing what you need to know and learning it.

Relying on others is fine to start out, but if you can't learn on your own you'll never be more than a pale imitation of your mentor. (or mentar, for that matter).

[Meltingice]

Originally Posted by cyth

Your friends must be pretty lucky to have you.

Of course.

Originally Posted by Quarkboy

Being a good encoder is like being a good scientist.

It's not what you know, but how good you are at knowing what you need to know and learning it.

Relying on others is fine to start out, but if you can't learn on your own you'll never be more than a pale imitation of your mentor. (or mentar, for that matter).

Well, this thread's topic is "for total newbies." It's significantly easier to have someone help you rather than to read over 9000 documents with no guarantee that it'll even answer your question to begin with. (No, I didn't learn from Mentar if that's what you were implying and the encoder I learned from retired ages ago.)

[Animeruko]

I'm not an "encoder" or a "fansubber" but what I do know i learned from experimentation and reading the provided documentation. Personally I think its the best way to learn something instead of having to always go ask someone something when an issue comes up, There are instances where this is actually necessary but not with encoding.