Question to Encoders: File Sizes getting out of hand?

[dj_tjerk]

Originally Posted by Hestia

Regardless, I think we might be missing the point here. It doesn't matter if my connection is good enough to easily hit several Mbps on an average day. The point is that I'm lucky to hit a 10-15 Kbps download speed for any semi-popular anime series a few months after it's completed due to a lack of seeders. So filesize does matter, especially if it means I'll get to download an episode in only half an afternoon instead of the next day.

I think that problem is more or less inherent to the way the bittorrent protocol works. I'm not really an expert on this matter in any way, so maybe someone else wants to elaborate on this matter. Based on experience and some papers on p2p however, I'll say that you might wanna try leeching the series from irc bots.

[martino]

Originally Posted by 0utf0xZer0

The only time file size bugs me is when groups do stupid things like making a 26 episode series 70MB too large to fit on a single DVD-R. And most of those groups use constant files sizes - what the hell.

Put the one that doesn't fit on a CD and sorted. Or use dual layer DVDs.

[neothe0ne]

Originally Posted by Tofusensei

I know it's an old concept, but some groups did that to discourage archiving in the hopes you may actually buy the DVDs when it gets licensed. I know that's why Hikari no Kiseki always used 180 megs (or whatever it was). Obviously it's more of a symbolic gesture than anything.

Problem with this logic is that many series have much better broadcasts than DVD's. See Hayate no Gotoku! and Zettai Karen Children in particular, although Toradora! also has a large amount of gg-esque grain. In fact, in the case of Zettai Karen Children, the DVD is so bad at times that I can't imagine trying to watch it on a large TV (it already looks like shit on my 22" monitor).

[Tofusensei]

Hence why it's an old concept.

[DryFire]

Except pirating and or giving away of infinite goods tends to help the sale of scarce goods...

Originally Posted by 0utf0xZer0

I actually still use DVD-R as my primary means of storing stuff because there's not way in hell I'd trust my collection to a single hard drive and I can't afford a redundant setup.

If you look around a bit you can get get 1 TB drives for ~$80 a piece. Put 3 or 4 into a RAID 5 array and profit. Most decent motherboards can handle raid 5 (not sure how well). That amounts to ~$.12/GB of usable space (rated) with a 1 disk failure tolerance.

That is about 2x more than the cheapest DVD-R price I found--it ends up being about $.05/GB - $.06/GB depending on shipping etc. However, that does not included bad disks, failed burns, storage space and the hassle of burning a bunch of DVD's.

[comatose]

The 175MB ones in this case look fine for the most part, but in the vast majority of cases it doesn't work this well =\ I think this is just a particularly good source.

[Kristen]

Originally Posted by comatose

The 175MB ones in this case look fine for the most part, but in the vast majority of cases it doesn't work this well =\ I think this is just a particularly good source.

Yeah, it's BS-J, a really great source. When I get K-On tomorrow or Friday I'll do some more tests off that.

[0utf0xZer0]

Originally Posted by DryFire

If you look around a bit you can get get 1 TB drives for ~$80 a piece. Put 3 or 4 into a RAID 5 array and profit. Most decent motherboards can handle raid 5 (not sure how well). That amounts to ~$.12/GB of usable space (rated) with a 1 disk failure tolerance.

I could afford to do something like this if I didn't buy R1 discs. ORZ

(Since I'm Canadian and our hardware prices are a bit different, my comparison would be $387 CAD (3 WD 1TB drives plus RAID 5 card) vs. $160 (5 100 packs of TDK DVD-Rs (95% success rate on burns with my drive) and 2 256 disc CD binders to store them). Would barely fit in my computer case too.)

Was kind of hoping someone would comment on the idea that variable file size could result in smaller overall series size than fixed size. Seems like it kind of makes the "constant file size to fit on DVD-R thing irrelevant.

[dj_tjerk]

Originally Posted by 0utf0xZer0

Was kind of hoping someone would comment on the idea that variable file size could result in smaller overall series size than fixed size. Seems like it kind of makes the "constant file size to fit on DVD-R thing irrelevant.

Of course it could, but it could also end up bigger. If one chooses a constant filesize based on the compressibility of ep1 (with a crf encode), and ep1 is like ultra compressible, but the other eps have high motion and are less compressible, the series would end up smaller with constant filesize than it would have with crf. It would also mean it'd look shittier however.

The other situation where ep1 is barely compressible and the rest is, and doing the same as the above, of course means that in the end crf will be smaller overall.

[DmonHiro]

Originally Posted by Hestia

The 175 MB images do have some extra artifacts, but other than that, the difference is minimal. And, minor banding aside, they all look good enough to my inexperienced eyes.

Well it looks AWFUL in my experienced eyes. ZKC needs 350 for all the high-motion-psychic power battles. Those looked bad at 233 and would look worse at 175.

[Daiz]

Originally Posted by 0utf0xZer0

That said, am I correct in thinking that this CRF thing produces smaller file sizes for the same quality level?

No, CRF and 2-pass at the same settings are pretty much identical at the same bitrate. That said, with CRF the episode filesizes will not be constant - you can never be sure what the final size for the entire series will be. The benefit of using CRF is that episodes that compress better will be smaller and the episodes that compress worse will be bigger. If you had the same series at the same size encoded as both CRF and constant-size 2-pass, the CRF version would be better since the episodes that compress worse would get the bitrate they need.

Another thing that can be used to reduce filesizes is obviously ordered chapters and segment linking. For example, take a look at these DVDrips for a certain series I made a while ago:



The filesizes for the episodes varies from 246 MB to 346 MB. The whole series is 3,4 GB. If each episode had the OP and ED in them, they would each be 52,6 MB bigger (except for the final episode that has unique animation for the ED, the episode would be 30 MB bigger) making the whole size of the series 4 GB. If each episode had been encoded as constant filesize around 279 MB (resulting in the same overall size) half of the episodes would have looked worse than what they look now due to the lower bitrate.

Of course you might think that 3,4 GB is big for a 12-episode series, but in my opinion DVDrips deserve high bitrates. DVDs are a lot better source than what you might think from some DVDrips - in general if you see notable banding / blocking in a DVDrip it most likely wasn't there in the original DVDs.

[neshru]

Originally Posted by martino

Put the one that doesn't fit on a CD and sorted. Or use dual layer DVDs.

Of course the point of his post is that he wants all the episodes to fit on the same DVD. And with a HD series that is 26 episodes-long, even a dual layer DVD may not be enough to fit all the episodes on one DVD.
Anyways, it's something that bothers me too. When it happens, I simply start cutting out previews or endings from the episodes until the total size is small enough.

[Dark Shikari]

Originally Posted by neshru

Of course the point of his post is that he wants all the episodes to fit on the same DVD. And with a HD series that is 26 episodes-long, even a dual layer DVD may not be enough to fit all the episodes on one DVD.
Anyways, it's something that bothers me too. When it happens, I simply start cutting out previews or endings from the episodes until the total size is small enough.

1. Run 1-pass, fast-settings CRF encode on all episodes.
2. Sum up file sizes. This sum is A. Write down all the bitrates for each episode.
3. Calculate the size of your DVD. This size is B. Maybe make it a tiny bit smaller so you have room for error.
4. For each episode, let NewBitrate = CRFBitrate * (B/A).
5. Run second pass on each episode.

There, magic, everything fits on a DVD but you still get proper bitrate distribution.

[TheFluff]

or you just make an avisynth script that concatenates all the episodes and encode the entire thing at once with 2-pass and then split with mkvmerge afterwards

[fireshark]

I wouldn't want to spend my time deciding split points 26 times in a 4gb file.

[TheFluff]

write a script that does it automatically based on individual episode script lengths and/or chapter points, should be fairly easy

especially if you already have a similar script that splits ts-extracted aac for you with mkvmerge based on a yatta-generated avs

[idiffer]

why are people fixated on getting 1 show on 1 dvd. does it really matter? all im concerned about is that the filesize is as small as possible with not that much quality loss. if a show takes up 3/4 of the dvd i'll just add part of another show. and are you supposed to freak when you see Naruto on your 10 dvd's?

and my reply to the linked segment thing...
i've noticed fansubbers tend to have that "i dont give a damn attitude", so they just do what they think is right or convenient for them. and i dont think thats wrong. so yeah, if linked segments is a hassle, why do it? i just didnt know it was hard or time consuming or whatever. so the linux and different credits arguements aren't that heavy...

PS. dvd's in my country cost a lot. i'm not willing to save money for a 500 GB drive, for like 2 months. i want to just buy packs of dvd's when i need them. + its safer that way. i dont get what that raid 5 thing is, but i still dont see how it solves the problem.

[neshru]

Originally Posted by Dark Shikari

1. Run 1-pass, fast-settings CRF encode on all episodes.
2. Sum up file sizes. This sum is A. Write down all the bitrates for each episode.
3. Calculate the size of your DVD. This size is B. Maybe make it a tiny bit smaller so you have room for error.
4. For each episode, let NewBitrate = CRFBitrate * (B/A).
5. Run second pass on each episode.

There, magic, everything fits on a DVD but you still get proper bitrate distribution.

I'm no encoder you know, I'm talking about series released by groups

Originally Posted by idiffer

why are people fixated on getting 1 show on 1 dvd. does it really matter?

It matters to some people, for whatever reason. For me, I'm just too lazy to switch DVDs while rewatching an entire series.

[Kristen]

Originally Posted by idiffer

why are people fixated on getting 1 show on 1 dvd. does it really matter?

It doesn't matter at all. DVD space isn't limited to only containing 1 series. The only reason why people like it on 1 DVD is so that it is easier to catalog. If I write K-On on one DVD, it's better than writing K-On on one DVD, and then K-On 12/Clannad 1-12 on another. Easier to navigate and put into a catalog.

That's the only reason I'd find for it. Honestly, though, I think you should aim for the smallest filesize while retaining quality, not aiming for a specific size that would either be too much or too little bitrate for what it needs.

[Zerox20]

Most groups just look at how many total episodes it is and use that to base there filesizes too I think.

For example if its a 26 Episode Series, and the source is a transport stream, well you can get away with ~350mb HD encodes, then you get 2 DVDs out of that 13 Episodes a disc so it works perfectly.

I would think depending on the series length would have some factor in file sizes too, cause in the end I would think people want to support the show and make people buy the actual DVDs when they get licensed.