Question to Encoders: File Sizes getting out of hand?

[Daiz]

From my experiments the higher the resolution the higher the CRF can be without notable quality loss. I've found that around 15-19 works for SD material, while anything from 19-23 will still look plenty good for HD material.

[comatose]

One thing I noticed is that a cleaner looking source (and I'm not talking about noise), like a Blu-ray, almost always compresses better than something off HDTV, even if the broadcast looks awesome.

Maybe it's the short parts that artifact on the broadcast, and in some shows you also have telecine fail here and there... it's just more stable, I guess.
Even if the Blu-ray has more detail than the broadcast, it still almost always compresses better.

And yeah Daiz, agreed. I don't go that low (never went below 18), but I did notice that HD sometimes allows for a higher CRF.

Originally Posted by Wadjet

Again I'll just say I am not trolling, I just wanted to know how todays encoders are viewing the subject.

Not every group releases an SD or LQ version, and sometimes the LQ version of some groups really are LQ. Thats why I asked about the codec.

I thought that the better the codec the smaller the file sizes, but it just seems that the sizes are just getting bigger.

I just want to know if the encoder mentality now is to try to hit a certian file size, or whatever comes out thats what it is. It's true that storage is cheaper nowadays and bandwidth is faster, but let's not forget some people have bandwidth caps.

Please don't flame I am just trying to gauge today's mentality. I know back in the day (yes 1996 ) encoders took pride in having the smallest file sizes and best quality

I think in the past, everyone just tried to hit 170MB. Nowadays, most people aim for perfect quality... and it usually ends up a size they consider manageable.
If it's bigger than 350MB (720p) I'll attempt to reduce quality a little and see if it's an acceptable loss. If not, I'll say screw it.

Encoders today still take pride in having the smallest file sizes and the best quality, but the bar for quality is higher. Most old encodes aren't too pleasing.

[DryFire]

A lot of broadcasts have some fairly ugly scene changes (Soul Eater, Eden) or the high motion frames just look like garbage (Michiko) which doesn't help compression. Hopefully all of that is rectified in BD/DVD releases.

I don't think most people notice, but it seems fairly common to some degree.

[idiffer]

why dont many encoders use chapters then? it reduces the overall size of the show.

[dj_tjerk]

I don't think you mean chapters, because that's just for making it easy to navigate through the episode and doesnt reduce the overall size of the show in any way. Anyway, I'm against using segment linking for tv fansubs, because you never know what they might change and many people just download one ep, watch it, and delete it (so they don't want to store some segment linked OP and ED, nor do they care about the overall size of the show). Also, support on linux.

For dvd releases, just do whatever the hell you want.

[Kristen]

Originally Posted by idiffer

why dont many encoders use chapters then? it reduces the overall size of the show.

There have been a few starting to use ordered chapters over the past few seasons. But, encoders don't use it for a few reasons.
1. Credits may change episode to episode
2. It is a lot less compatible with non-windows platforms that don't support CCCP. I've heard of mods to allow for it, but it's not like that's as easily found/installed/updated as just DLing CCCP. So,there's little to no support on Macs and Linuxes.


Concerning the topic of this thread, I do agree that some of the file sizes are a bit unreasonable. I've done some encoding tests to check the recent "Bigger file sizes = better quality!" theory, and no, that is not the case. Like with K-On, I encoded it once to 100 MB as usual, and then to 150 MB. Displaying the same frame, they looked identical. And if you could spot an error, it'd be one you'd find with a magnifying glass in a freeze frame situation, not one you'd find when actually watching the show.
It seems that you don't notice a difference until the P-Quant goes above 17 in the error stream of h.264.
For 720p, there is a need for larger files, but some of the filesizes for 720p are a bit over the top. There should really never be a need to go about 350 MB, in my opinion. Especially not for upscales.

A lot of the larger filesizes have to do with fitting a certain amount of episodes onto a CD or a DVD, though. But with the advent of external hard drives, I really think episodes should start being encoded more to sizes that they deserve instead of added megabytes for no noticeable gain in quality. >.<

[comatose]

Like with K-On, I encoded it once to 100 MB as usual, and then to 150 MB. Displaying the same frame, they looked identical.

Color me skeptical. Well, post the frames you compared. Don't forget the same frame from the raw.

A lot of the larger filesizes have to do with fitting a certain amount of episodes onto a CD or a DVD, though. But with the advent of external hard drives, I really think episodes should start being encoded more to sizes that they deserve instead of added megabytes for no noticeable gain in quality. >.<

Actually, some 350MB encodes need more than that, but they're restricted to fit 350MB so they fit on a CD/DVD. I don't think too many people just blindly encode with the same bitrate if they see it's an episode that needs less.

[Daiz]

Obviously a frame can look identical compared to an encode with higher bitrate, but what you should compare is stuff like fades, high-motion sequences and other bitrate-hogging scenes like that. I've done similar comparisons where a frame can look basically identical but things like fades etc got totally fucked up.

EDIT: But all in all, everyone should just use CRF. I do almost exclusively CRF encodes these days and certainly don't miss the days of only doing 2-pass encoding.

[Kristen]

Originally Posted by comatose

Color me skeptical. Well, post the frames you compared. Don't forget the same frame from the raw.


Actually, some 350MB encodes need more than that, but they're restricted to fit 350MB so they fit on a CD/DVD. I don't think too many people just blindly encode with the same bitrate if they see it's an episode that needs less.

Heh. I deleted them 2 weeks ago. I can make a new test on Friday if it can wait that long.

[Dark Shikari]

Originally Posted by dj_tjerk

I don't think CRF18 is that high, but then again I might be using totally different settings (I don't like using psyrdo for most anime). Also, how does crf scale with resolution?

Since you're watching it all on the same size screen, you can get away with higher CRFs at higher resolutions.

Originally Posted by Daiz

Obviously a frame can look identical compared to an encode with higher bitrate, but what you should compare is stuff like fades, high-motion sequences and other bitrate-hogging scenes like that. I've done similar comparisons where a frame can look basically identical but things like fades etc got totally fucked up.

Raise qcomp if you think that's a problem.

[comatose]

So, what are the negative effects of raising qcomp?

[Daiz]

Dark_Shikari, just to clarify, the test was basically a 100MB 2-pass encode versus a 180MB 2-pass encode of the same thing. A very low-complex frame on a very static scene looking basically identical in both wasn't that surprising. In any case, this was ages ago and with a shitty source to begin with.

I've been experimenting with qcomp, psy-trellis and CRF lately though. Interesting stuff.

[Dark Shikari]

Originally Posted by comatose

So, what are the negative effects of raising qcomp?

x264 believes that more complex scenes deserve higher quantizers than less complex ones.

You may disagree, or believe that x264 is too aggressive at this, in which case you may raise qcomp.

[martino]

Originally Posted by Kristen

But, encoders don't use it for a few reasons.
1. Credits may change episode to episode

I highly doubt this point applies, because the same goes for DVDs, and doesn't seem like most encoders care there. That is, unless they leave the ED in the episodes (much less likely that the credits will change in the OP). Or yay for clean OP/ED.

[comatose]

That's probably just an excuse to avoid the trouble of ordered chapters :P

[Waryas]

Ordered chapter is nice and all but should be avoided if you can get reasonable QP/Quality within an acceptable range of size.

In my case if the show looks meh @ 450MB then yea i'd use ordered chapter to save bitrate but for anything below that, screw it, we aren't in 2000 anymore : HD space is dirt cheap.

Even my eeepc can play a 400MB HD encode fine with CoreAVC so there's no excuse.

[Kristen]

Originally Posted by martino

I highly doubt this point applies, because the same goes for DVDs, and doesn't seem like most encoders care there. That is, unless they leave the ED in the episodes (much less likely that the credits will change in the OP). Or yay for clean OP/ED.

Yeah, I don't either. Only time I've seen someone complain was when the ZKC OP changed to say that "My Wings" was featuring The Children instead of Karen's Girls, and Karen's Girls was in the release even though the audio changed.

Though with DVDs, groups seem to like to do NC OP/ED.


Personally, I like ordered chapters and want to use them more on releases. Easier seeding, distro, and DL. ^.^

[neothe0ne]

Originally Posted by Kristen

Yeah, I don't either. Only time I've seen someone complain was when the ZKC OP changed to say that "My Wings" was featuring The Children instead of Karen's Girls, and Karen's Girls was in the release even though the audio changed.

Hello to you too, Kristen!

Fact is, most leechers don't even give two cents about translation, so I imagine you would be hard pressed to find anyone else complaining about wrong credits in releases. But on the other hand, few groups actually hardcode a karaoke song sequence and then splice it into all their releases. I can say this with certainty because the timing of karaoke can be noticeably different over a season with the same group - in fact sometimes it is so horribly off that I have to wonder if QC existed in those groups.

Ultimately this topic comes down to a simple reality: if the group is offering something the fans want, they will keep downloading. A significant portion of the community would prefer clean, HD encodes, or else encoding 500mb episodes would be a waste of everyone's bandwidth. Although I also agree with many in this topic that everyone (including Otaku's Dream) should stop adhering to filesizes and do quality-based encoding instead.

[Kristen]

Originally Posted by neothe0ne

Hello to you too, Kristen!

Fact is, most leechers don't even give two cents about translation, so I imagine you would be hard pressed to find anyone else complaining about wrong credits in releases. But on the other hand, few groups actually hardcode a karaoke song sequence and then splice it into all their releases. I can say this with certainty because the timing of karaoke can be noticeably different over a season with the same group - in fact sometimes it is so horribly off that I have to wonder if QC existed in those groups.

Ultimately this topic comes down to a simple reality: if the group is offering something the fans want, they will keep downloading. A significant portion of the community would prefer clean, HD encodes, or else encoding 500mb episodes would be a waste of everyone's bandwidth. Although I also agree with many in this topic that everyone (including Otaku's Dream) should stop adhering to filesizes and do quality-based encoding instead.

Hence why it's a rarity.

I just happened to come across today when I needed to encode Zettai Karen Children 05, so I'm going to encode it to 350 MB and 500 MB, and grab a high-motion screenshot to compare. It's a good show to do it on since it's high motion and 29.970 fps.

Though it shouldn't be able to show that there's no noticeable difference between quant 17 and quant 14 (I usually get like 18 or 19 at 350 MB), it should give a glance into the differences between 350 MB and 500 MB.

[Heibi]

You know, after reading this thread, all you people should just be happy that the file size isn't ZERO. i.e. no FREE fansubs. Remember, the people doing the translation, editing, timing, and encoding aren't really getting paid. So sit back, pop a beer, soda, wine bottle or favorite beverage and enjoy the FREE show.