Question to Encoders: File Sizes getting out of hand?

[Zero1]

Ah, the age old debate. A few key words: Diminishing Returns and Quantization.

Diminishing returns means exactly that. You can have a 175MB encode that may look Ok. Increase the filesize or quantizer by an amount (eg 50MB or q2) and you will notice an improvement. Increase it again by the same amount and you will notice another improvement, but the quality difference between the first and the second and the second and the third will be less. You increase the filesize by the same amount each time, but the quality does not increase by the same amount.

Without going into ye olde long poste mode, the basics of this are high frequency coefficients, or basically sharp lines or detail within an image that are transformed. The sharper an image is or the more detail it has, the more high frequency coefficients it will have/produce, and these typically require more data to be coded. At a low quantizer, much of these high frequency coefficients are zeroed out. If you raise the quantizer, it is able to retain more of these, meaning that one encode will have more detail than the other. This can go on and on, but there gradually comes a point where it retains more and more high frequencies that the increase of detail is hardly noticeable by the human eye under normal conditions (for example not zooming in or studying a still image).

So you have diminishing returns on two levels. One in the filesize vs detail retained and the other in detail retained vs detail appreciated by the viewer. Obviously some people have better eyes or better setups than others, so it's subjective, as is most aspects of encoding.

Loosely related:
http://forums.animesuki.com/showthre...567#post905567
As I go on to explain, this is why you get different quantization matrices. Some are tuned more for low bitrate, and as such discard more high frequencies after quantization, other are designed for quality and retain more high frequencies.

[Dark Shikari]

Originally Posted by Zero1

Loosely related:
http://forums.animesuki.com/showthre...567#post905567
As I go on to explain, this is why you get different quantization matrices. Some are tuned more for low bitrate, and as such discard more high frequencies after quantization, other are designed for quality and retain more high frequencies.

The primary purpose of quantization matrices is to work around the weaknesses of MPEG-4 ASP, particularly the horrific lack of quantizer precision around the QP2-4 mark. In terms of zeroing high frequencies, the encoder itself does a rather good job at that regardless. You don't need CQMs.

In particular, the DCT of an edge is a Laplacian coefficient distribution from low to high, so naturally all coefficients beyond a certain point get zeroed.

Additionally, my experience shows that the high frequency coefficients contribute less to sharpness than one would think. Visual sharpness not only depends on the actual sharpness of the boundary between dark and light, but also the contrast. As a potential psy optimization a few weeks back, I tried implementing the basic concept of CSF masking. The basic idea is that larger DCT coefficients tend to require less precision, so you can get away with rounding them down since the relative error is lower (the relative difference between 9 and 8 is much smaller than between 2 and 1). So, in trellis, I weighted the value of various DCT coefficients (larger values were lower weighted).

... the results were worse. In particular, the darkness of the edges was reduced, making them seem more blurred even though the actual boundaries of the edges might have been sharper. This makes me suspect that the logic of "high frequencies mean sharpness" is not always sound.

[deadman80]

As I am not one who is actively involved in the subbing side of fansubbing, can anyone tell me what the reasoning is behind releasing a 12 episode series that is 1MB to large to fit onto a single DVD, if there is one?

I have recently run into numerous series that were all just a tiny bit too large (100KB - 40MB) to fit onto the deceptively named 4.7GB standard DVD-R -_-;;;

tangential mini-rant: Someone should track down the person responsible for the decision to allow data storage media manufacturers to round down to the nearest base10 number and use the binary name in packaging, then slap him/her around with a large trout. Every 1GB of advertised storage space is short by slightly more than 70MB... End result: a "4.7GB" DVD = 4,595,776 KB =/= 4.7GB -_-;;;;;;;;

Back to my thread topic related question (^^;, is it possible for a layman like myself to use any of the standard fansubbing software out there to reverse engineer an .mkv and reduce the filesize slightly on my own?
Recently, I have been experimenting with Aegis Sub to correct typos and grammatical errors of fansubs for my own use, and I was planning to teach myself how to time subs as well. I am not adverse to learning new software in order to achieve my goal of satisfying my compulsive organization and collecting/archiving tendancies. XD

[TheFluff]

Originally Posted by deadman80

As I am not one who is actively involved in the subbing side of fansubbing, can anyone tell me what the reasoning is behind releasing a 12 episode series that is 1MB to large to fit onto a single DVD, if there is one?

I have recently run into numerous series that were all just a tiny bit too large (100KB - 40MB) to fit onto the deceptively named 4.7GB standard DVD-R -_-;;;

if your dvd burning software and/or hardware doesn't supporting overburning at least like 20mb or so it really sucks a lot and you should seriously consider upgrading

Originally Posted by deadman80

Back to my thread topic related question (^^;, is it possible for a layman like myself to use any of the standard fansubbing software out there to reverse engineer an .mkv and reduce the filesize slightly on my own?

you could always reencode the audio I guess if you're just missing <50mb on an entire series... use mkvextract and your favorite audio conversion tool.
you probably don't want to reencode the video as it is kind of a pain and takes a longass time to do.

[Miles Teg]

Originally Posted by deadman80

tangential mini-rant: Someone should track down the person responsible for the decision to allow data storage media manufacturers to round down to the nearest base10 number and use the binary name in packaging, then slap him/her around with a large trout. Every 1GB of advertised storage space is short by slightly more than 70MB... End result: a "4.7GB" DVD = 4,595,776 KB =/= 4.7GB -_-;;;;;;;;

GB is base 10, if you want to speak in base 2 then use GiB (what manufacturer won't do). DVD 4.7GB=4.38GiB, Blu-Ray 50GB=46.6GiB.

If the OS where to use the GB like it should be then nobody wouldn't be complaining about the size difference

[Scab]

Well, this is kind of off topic, but to be somewhat fair the prefixes for binary multiples were introduced after regular kilo-style prefixes became the standard and no one has felt much like adopting new prefixes that the average consumer isn't already familiar with. As confusing as this has become, neither party is technically "in error".

[Isohunter]

I think the 340-500MB is perfectly reasonable size for 720p, anything lower than that is just crap to me.

I archive the shows I download (can't buy them) and 340-500MD per episode suits my taste just fine.

[fireshark]

I think we should all release TS and remux in subs as actual CC.

[Quarkboy]

Originally Posted by fireshark

I think we should all release TS and remux in subs as actual CC.

Screw that, we should go back to trading tapes! Uncompressed HDCam 1080p tapes, that is. 200 GB an episode~

[Schneizel]

I think you should all quit making comments like that and actually do it. (Same goes for the people in the karaoke thread who talk about Flash and whatever.) If we went through a whole phase where everyone encoded retardedly huge files, it would make the whiners appreciate the old (now current) file sizes more.

[idiffer]

Originally Posted by Schneizel

I think you should all quit making comments like that and actually do it. (Same goes for the people in the karaoke thread who talk about Flash and whatever.) If we went through a whole phase where everyone encoded retardedly huge files, it would make the whiners appreciate the old (now current) file sizes more.

that's a serious statement! someone can actually do that once or twice for the lulz, but after that it would look stupid.
oohh, karaoke thread...should go look for this fail thread.
you actually made me go out of hiding and reply, dammit. and thats a very serious thing to do!

[jfs]

So... a flash video encoded as lossless H.264 1080p with PCM 5.1 audio and subtitles + karaoke as flash objects?

[Daiz]

So, 26GB lossless 1080p files?

[fireshark]

and release in rar

[Dark Shikari]

Originally Posted by jfs

So... a flash video encoded as lossless H.264 1080p with PCM 5.1 audio and subtitles + karaoke as flash objects?

Lossless H.264 doesn't work in Flash.

I prefer this.

[CeDeR]

well i have a new 1.0tb hard disk so i no longer care...also i enjoy watching animes at the max quality i can get since i play them on a full hd flat screen.

[Schneizel]

Anime [Chihiro]​_Suzumiya​_Haruhi​_no​_Yuuutsu​_(2009)​_-​_08​_[1440x1080​_MPEG-2​_Transport​_Stream​_AAC][81BAD57D].mkv Website | Details
Size: 1.75GiB | Date: 2009-08-03 19:25 UTC | Comment: Encoding is so old fashioned. #Chihiro@irc.rizon.net

I AM SO PROUD OF YOU CHIHIRO.

Tears of JOY. ;_____;

(Although I think the CRC should have been brute forced to all 8's in honor of ENDLESS EIGHT.)

[Kristen]

http://i25.tinypic.com/5ytfdd.png
http://i30.tinypic.com/zn6emp.png

http://i29.tinypic.com/bijvc0.png
http://i25.tinypic.com/28s92l5.png

Since I failed to remember to resize, here's another sample difference between the same screen at 350 MB and 175 MB.

[grumble]At least now this failed encode has some value[/grumble]

[Zergrinch]

I can barely make out the difference. I sincerely hope the first picture is the lower-quality version. If it's not then I'd probably need to get my eyes checked

[jorez]

remember when single anime eps were only 175MB? those were the days


Leaving, that aside, I think 340MB is fine. But 400MB?! I mean, it's nice that they created dual layer DVDs for archivefags like me, but... help me a bit here, encoderman ;_;