2007-02-09, 01:20 | Link #721 |
Translator, Producer
Join Date: Nov 2003
Location: Tokyo, Japan
Age: 44
|
It should also be pointed out that --aq options DO NOT improve compression.
They rearrange the available bits to make compression artifacts less noticeable. So although using them slows things down, whether or not they actually improve the quality of the encode is very very source (and sometimes, viewer) dependent.
__________________
|
2007-02-09, 06:43 | Link #722 |
Excessively jovial fellow
Join Date: Dec 2005
Location: ISDB-T
Age: 37
|
ATTENTION PEOPLE
It's really really stupid to use ref=1. Multiref is one of the big bitrate savers in h264 and YOU SHOULD USE IT. However, it's also pretty pointless to use ref=16. Anyway x264 itself tells you what to do if you only take time to read what it's saying. Example from my last encode (I used ref=12 because I'm anal retentive): Code:
avis [info]: 704x400 @ 29.97 fps (42548 frames) x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow! x264 [info]: slice I:335 Avg QP:15.65 size: 27088 PSNR Mean Y:51.13 U:53.80 V:53.30 Avg:51.71 Global:51.04 x264 [info]: slice P:24488 Avg QP:17.66 size: 3761 PSNR Mean Y:48.60 U:51.93 V:51.60 Avg:49.36 Global:48.58 x264 [info]: slice B:17725 Avg QP:19.04 size: 1200 PSNR Mean Y:48.35 U:51.52 V:51.24 Avg:49.08 Global:48.19 x264 [info]: mb I I16..4: 21.2% 29.6% 49.2% x264 [info]: mb P I16..4: 2.9% 2.7% 1.6% P16..4: 32.3% 5.1% 4.1% 0.4% 0.3% skip:50.7% x264 [info]: mb B I16..4: 0.1% 0.1% 0.1% B16..8: 15.5% 0.8% 2.2% direct: 1.1% skip:80.0% x264 [info]: 8x8 transform intra:36.4% inter:61.7% x264 [info]: direct mvs spatial:88.6% temporal:11.4% x264 [info]: ref P 77.4% 9.0% 4.8% 1.8% 1.5% 1.2% 1.2% 0.7% 0.6% 0.5% 0.5% 0.6% x264 [info]: ref B 74.7% 10.3% 4.2% 2.4% 1.8% 1.5% 1.2% 1.0% 0.8% 0.8% 0.6% 0.9% x264 [info]: SSIM Mean Y:0.9938808 x264 [info]: PSNR Mean Y:48.517 U:51.774 V:51.461 Avg:49.264 Global:48.428 kb/s:690.07 encoded 42548 frames, 4.35 fps, 690.15 kb/s You may now draw your own conclusions about how many references you want to use.
__________________
|
2007-02-09, 10:26 | Link #724 |
Part 8
IT Support
|
They're basically a kludge to work around the problems x264 has with dark scenes. All it does is increase the bitrate allocated to flat blocks, ie blocks with little detail. Since these blocks are often where x264 falls over, it's a reasonably effective to fix.
Not it's not part of normal x264, it's a custom patch that sharktooth compiles into his builds. Read the --longhelp for a little more info. |
2007-02-09, 18:38 | Link #725 |
Junior Member
Join Date: Mar 2006
|
That CRF 18 profile on meGUI use 3 B-frames and 3 Mixed Refs maybe it is better to decide the amount of refs looking at how much B-Frames we are going to use?
2 B-frames -> 2 Mixed Refs 3 B-frames -> 3 Mixed Refs 6 B-frames -> 6 Mixed Refs Or it is wrong to think like this? |
2007-02-09, 19:07 | Link #726 | |
Translator, Producer
Join Date: Nov 2003
Location: Tokyo, Japan
Age: 44
|
Quote:
I.e. although it might be true that sources which support large numbers of b-frames also benefit greatly from larger refs, they don't have anything directly to do with each other. Recall that the number of b-frames controls the maximum number of consecutive b-frames that could be used. If a particular frame references a frame from, say 8 frames before with a mixed refs = 8 setting, you might say that it would be most efficient if you had 8 b-frames, letting all 8 frames in between be B and save bitrate. However, perhaps the motion is oscillatory, and the intervening frames are, in fact, not related to the boundary ones... Then b-frames wouldn't be appropriate, but the references would. It's a complicated picture indeed. Plus, there are good reasons NOT to use large numbers of b-frames like 8 or 9 (namely, it increases the needed data bandwidth and decoding complexity (lookahead)), even if the source could support it, but large numbers of references aren't as bad because you don't need to reconstruct the entire frame, only individual macroblocks... (Note: I'm not completely sure of this fact. It's speculation based on my knowledge of how h.264 compression actually works with mixed references. Perhaps someone on doom9 might know more specifics.). So in conclusion, the answer to your question is : I don't think there's any particular logic behind that reasoning, but it might work as a rule of thumb.
__________________
|
|
2007-02-26, 17:37 | Link #730 |
Two bit encoder
Fansubber
Join Date: Jan 2006
Location: Chesterfield, UK
Age: 39
|
So while I sit here on family tech support, fixing peoples mistakes (aka I magically killed Windows and need you to fix it), recovering shit from some archaic computer using a Knoppix live CD (praise be to opensource devs everywhere) and using a HDD Walkman as an external harddrive because I'm so out of space; I thought I might try to provoke some thoughts. It's something I was thinking the other day, having attempted to distance myself from encoding/fansubbing, move back to gaming, failing and somehow getting roped back into it all.
It's been around 4 months since I decided to move back gaming, and I thought to myself, "Where is the scene now?" This of course can be interpreted in many ways; what's hot/what's not, is oversubbing/speedsubbing still an issue, what's the latest drama; but what I was getting at is the ASP to H.264 transition, and also AVI to MP4/MKV. Here's a few things to ponder. 1) Are you happy with the way things are going; ie do you think it's taken fair time, or too damn long? 2) What do you see as obstacles (if any) to the adoption of H.264 and/or modern containers? 3) How do you think you/we can assist adoption, if possible? Are you doing anything? Are you bothered? What are you doing? 4) Do you ever forsee a total switch to the newer formats, as we did with DivX 3.11 > XviD (SP to ASP), or do you think that with AVI having been the defacto standard for so long, that it will simply co-exist? 5) Do you think there should be an indefinite transition period, or should groups/scene somewhat agree on a cutoff date of sorts (like how terrestrial TV will soon be shut down leaving digital only)? I perhaps already know what a few people will already say, but to answer some of my own questions: 1) No, I'm not happy with the transition, and it's taken too long compared with the previous transition, granted that this time there are new containers to deal with also, yet it shouldn't be an issue. I've been using x264 for almost 2 years now, give or take a month or two; and I'm sure I wasn't the first to make the change. Also it's probably fair to say that x264 has offered advantages over XviD for 2 years or more now, so it begs the question, "Why change only now?" 2) I cannot single out one thing, so if I choose something fansubber side it would be the familiarity/ease of use of Virtualdub (and ultimately AVI; Virtualdub is not to blame, it's just people are so attached to it and VfW). Leecher side, I say the one of the bigger issues is file extension. People managed years back when the transition was between MPEG-4 SP (or MS'/DivX bastardisation of it) and ASP in AVI, since you were only required to install a codec. The files are still associated to whatever software it already was. Perhaps you got some people whining about CPU usage requirements (what with the addition of B-frames, QPEL, GMC), but on the whole it wasn't much of a task to install one video decoder. Now you have the issue that you require a video decoder, audio decoder and splitter (in most cases); and also that double clicking files doesn't make them play in your favourite player, the horror! (so perhaps add a registry entry, or some manual association to that loooong list of requirements). This is what I see as one of the issues, it's no longer as simple as installing one decoder, you now install two decoders, a splitter and associate the files (which is all a one off and maintenance is very low to none), fortunately however, it seems that other encoders noticed this, and as a result, CCCP exists. They are now given an easy, single installer that performs those tasks for them so now they only have to execute one installer, just like back in the AVI days. 3) I have pimped H.264 to a few groups I have been involved with, also pimped it to some individuals I hang around with who tend to edit AMVs, also written encoding and playback guides and everyone that has made the change loves it and would not look back. They are that impressed with the quality and filesizes. I have also pimped CCCP, helped out on occasion, and of course longposted in legendary threads. I don't want to give myself a big head, but I've done what I can to basically promote encoding H.264 and point out easy playback methods; so yes I am bothered. I enjoy encoding, and these are interesting times. The previous transition wasn't all that exciting because it was merely moving from SP to ASP, but this time we are eventually ditching AVI also, so it's double the challenge/benefit. 4) Unfortunately my crystal ball is out of order at the moment. I don't really know what to expect. If I had to make some kind of prediction, I would guess that AVI would drag it's feet for many years as MP4/MKV grow even more popular, but I don't see it ever dying; in fact I expect it to linger indefinitely like MPG. Virtualdub, XviD, Lagarith and Co. are just too damn convenient to throw away for good. We will all be guilty of doing "quick and dirty" XviD's at some point in the future (eg previewing something). 5) As with the last question, this is difficult for me to answer. Personally, I think I'd like to see a set transition date, but there are a number of issues. First is "How much of a community is your community?". I think you'd have a hard to impossible time getting fansub groups to make a global transition. You see groups share the same standards when it suits them, such as everyone using XviD, MP3 and AVI, yet they cannot agree to, or work out something to lower oversubbing (or rather, attempts have been made but it still happens a lot). Even if a transition date were set and adhered to, there would always be opportunists popping up with new subbing groups, or transcoding groups fighting for the adoration of the leechers. In the end it could even be detrimental if transcoding groups were doing a substandard job. What I have found interesting with the transition so far is that there is no clear cut "scene standard" like there was with XviD, MP3 and AVI. It's not uncommon to find H.264 and/or ASP used with AAC and/or MP3 in MP4, or additionally Vorbis and/or AC3 in MKV. Perhaps this is due to us having more choice, now in part from us not being tied down due to AVI. Perhaps I have answered all my own questions a bit too much and no one will have anything to reply with On that note, it's back to tech support By the way, this machine I'm "fixing" seems to be a P3 950Mhz (Coppermine) with 256MB RAM. So anyone up for some non scientific testing?
__________________
|
2007-02-26, 19:28 | Link #731 |
I see what you did there!
Scanlator
|
With regard to scene standards, Xvid has been a solid standard for us since 2003. It brought sanity to a community that didn't really have any clear-cut solution or standard beforehand. Rather than a mess of Divx versions and variants, and WMV files, there was a single codec that a majority of encoders could agree on.
The adoption of Xvid had a bit of a "chicken or the egg" scenario because it coincided with the proliferation of BitTorrent and first-generation torrent indices. Fansubbing now had a far greater outreach and the releases formats of groups was now more more publicly visible. Encoders see more groups are jumping on the bandwagon and make the transition. However, the community is being real stubborn this time around. I don't see anything meaningful happening with regard to the transition unless there is a more organized community-wide assemblage.
__________________
Last edited by Starks; 2007-02-26 at 19:55. |
2007-02-26, 19:53 | Link #732 |
Aegisub dev
Join Date: Sep 2004
Location: Stockholm, Sweden
Age: 39
|
For just point 3:
I think one of the major obstacles might not be as much (perceived) technical problems as it's lack of perceived gain. If you ask a random viewer, chance is that they can't see much if any improvement from a CFR XviD encode to a CFR x264 encode, most people simply don't see a difference or don't care about it. "Why bother if I don't gain anything?" Div3 on the other hand had all kinds of odd problems that were easy to spot and were very clearly worse quality than what XviD delivers/delivered, and the XviD files even were smaller to deliver that better quality. So of course the viewers would switch there. Now what very obvious problems does H.264 in MP4/MKV solve that XviD in AVI doesn't already provide a solution to? Nothing anyone would really care about, current Xvid/AVI releases are already as watchable as things need to be, you need to be an encoder to really appreciate the gains. So a quick shot at point 3 might be in order, how about beginning to offer even smaller H.264 encodes than current XviD encodes while avoiding a quality drop? I believe that (at least for SD resolutions) that's a much more viable option than telling about all the invisible gains of same-sized H.264 encodes. And while at the size issue, I once again beg to drop those N-eps-per-CD/DVD file sizes. I believe most if not all of the people who archive anime have switched to DVD's now, so it'll generally not be as much (potentially) wasted space per disc when the file sizes fall. Of course there's still the "full anime on one or two discs" issue, but really... does it matter that much? It's nowhere near as bad as with CD's, with CD's you'd have to change disc all the time, with DVD's it's at most two disc changes for average series anyway. Uh, maybe that was a bit off-topic, mods feel free to split it away
__________________
|
2007-02-27, 02:59 | Link #733 |
Junior Member
Join Date: Mar 2006
|
With H264 it become stupid to release something at 175 MB like we always did with Xvid because there is a lot of animes that If we encode using CRF 18 the filesize will be near 140 MB or a little lower.
As you already have read on doom9 forum the CRF 18 is the same quality as the Xvid Q2 but the final filesize is really different than Xvid and we dont need things with lower quantizers because our eyes dont see the difference. When we do encodes with 175 MB for H264 video all quantizers is very low, the quality sure is very very good but we dont really need those lower quantizers, we can release everything with at last QP 20~22 that the image can still look very good and we can make files that is really small. With CRF 20 you can make things from 100 MB to 140 MB. With CRF 22 you can make things from 100 MB to 60 MB. There is a lot of animes that you dont really see any very good motion out of the opening and ending that need high bitrate and when you encode such a very slow anime you can see that the quantizers used is even lower than 18 soo your eyes will never see such a difference, you wasted a lot of bitrate for nothing soo a 175 MB dont really worth for such a slow anime. Im always downloading releases from the fansub Froth-Byte their releases have a very good quality and they dont use 175 MB files anymore and everyone that watch their releases really love the video and audio quality. That is the proof that it is time to drop those 175 MB encodes. We dont need this big filesize anymore we can make everything with lower filesize. Big filesizes is only needed for the HDTV releases because the imagem is more sharp and bigger soo those releases really need more bitrate. Here on my country we still use dial-up connections and slow ADSL connections is very expensive. There is a fansub here that release a lot of animes and make everything with 100 MB or lower and there is a huge amount of leechers that download those small files and everyone is happy with the quality and the smaller filesize because they can donwload it with their slow connections and watch on their old computers(1 Ghz~1.5 Ghz). If here we are adopting this way of using the H264/AAC codecs the people that make english fansubs can do it too without any problems. The leechers just need to know that with H264 we can make very small files and still have very good quality. We all already have computers that is able to play 704x400 and 640x480 H264/ACC/MKV/MP4 encodes soo we just need to teach the leechers how to install the codecs and splitters needed or just make those people install the CCCP package. The X264 encoder is the one that produce the best quality at the moment and is very stable soo we can really trust the X264 encoder. The AAC encoders is very good and they are still going to use even lower bitrates for audio encodes, at the moment we can already use a very low bitrate such a HE-AAC 64 kbps that have the same quality as the MP3 128 kbps that we have used for years. With HE-AAC you can have a very good quality for stereo audio with just 96 kbps that have the same quality as the MP3 192 kbps. You dont really see any big difference when you hear a 320 kbps and a 192 kbps MP3 encode soo that means we can use the max of 192 kbps for MP3 stereo audio and if we use AAC we can do encodes with LC-AAC 160 kbps or HE-AAC 96 kbps and get the same quality as the MP3 192 kbps. Using lower bitrate on the audio tracks means more bitrate for the video track soo you can get better quality for the video track or dont use this bitrate on the video track and get a small filesize. Even for a CRF 22 encode you dont need to use those very slow encode options like subme 7, trellis 2, exaustive, a lot of references and more than 1,1 deblocking. That means that you can still have a good encode speed even with small filesize. There is a lot of fansubs releasing two files one with Xvid and other with H264 but they are still doing it the wrong way because those two files have the same size, the H264 really have to be smaller than the Xvid encode. A small filesize will make the people get interest on H264. |
2007-02-27, 07:45 | Link #735 | |
Excessively jovial fellow
Join Date: Dec 2005
Location: ISDB-T
Age: 37
|
Quote:
Many shows tend to reach CRF18 around 130-160 MB (without audio). Of course, some reach it around 70-90 (hi2u Night Head Genesis), and some need 200+, but they're pretty rare. And since I'm a quality whore, well... I'd rather have CRF18 than 30 MB less to download.
__________________
|
|
2007-02-27, 10:29 | Link #736 | |
Part 8
IT Support
|
Quote:
Anyway, the real reason h264 is not being adopted is simply because for those who can't tell the difference in quality (sidenote: I can remember a time when I thought the quality of 175mb dvdrips was nigh on flawless), there are only reasons against using h264. The ONE advantage of h264 has been negated, and you end up with files that are essentially less useful than before. It's been said time and time again: if you want more people to start liking and even embracing h264, encode a 120mb version of your fansub and put it up on the tracker for download. The amount of extra effort to encode a second version at 640x480 500kbits plus a 96kbits AAC is minimal. The gains could be tremendous. |
|
2007-02-27, 11:55 | Link #737 |
Translator, Producer
Join Date: Nov 2003
Location: Tokyo, Japan
Age: 44
|
"Xvid=seatbelts, h.264 = hydrogen cells"
Might I propose a radical viewpoint: The Divx 3->Xvid transformation was a process of "maturity" The Xvid->h.264 transformation is a process of "generational shift". This is because of the reasons stated above, that the majority of fansub consumers don't see a radical enough benefit of going from xvid to h.264. The second reason is because of the filename extension shift. What matters most here is whether the leecher has had PREVIOUS EXPOSURE to it. This immediately breeds familiarity to it and makes it more likely for someone to download it. Once it's been downloaded, that's 80% of the battle over, as people will then (probably) put in the effort to install/configure in order to play it. That's another reason why I've decided to shift to .mp4 instead of .mkv (people know mp4s from ipods and psps and ps3s). So what I'm saying is that it's not so much a process of convincing a small number of people of something like the divx 3->xvid transition, but it's more a matter of waiting for enough NEW fansub watchers to come along and change the majority mindset. A good metaphor would be the auto industry: Seatbelts were initially resisted heavily, but once they started being installed (and saved tons of lives) the entire industry came onboard almost immediately. But when it comes to fuel efficiency and alternative energy powered vehicles, it's taking a generational shift to change that sort of thing. I.e. xvid=seatbelts, h.264 = hydrogen cells.
__________________
|
2007-02-28, 17:45 | Link #738 | |||
Two bit encoder
Fansubber
Join Date: Jan 2006
Location: Chesterfield, UK
Age: 39
|
Quote:
In fact I made a quick test to compress Lena with JPEG and JPEG2000, unfortunately the best JPEG could do was 37.48db, so I had to use that as my target (JPEG2000 was well over 70db at the higher end of it's quality scale). Now although PSNR is not the best quality metric ever, as an unscientific test, this was an interesting result. JPEG: 212498 bytes, 37.48db PSNR JPEG2000: 94250 bytes 37.53db PSNR So that's 92KB vs 207KB, that's a great reduction for such a filesize to begin with. The image files are available here: http://aflux.deltaanime.net/Zero1/lena.jpg http://aflux.deltaanime.net/Zero1/lena.jp2 It almost makes you wonder what the next wavelet based standard from MPEG will be like (having said that, SNOW can still reach very far). Another, familiar scenario would be Operating Systems. Windows has an incredibly huge userbase, and despite it's failings, or the fact that Linux or Mac may be faster/more stable/more user friendly/more fully featured/bullshit free; people stick with Windows because Windows is the main player. Most software, file formats and hardware work with, or are designed to work with Windows; some do not even have Linux or Mac counterparts, effectively limiting your choices and making Windows "indispensable". Again, people let the possible advantages slip because they want the easy ride, and the interoperability. They want to know that programs and drivers are readily available and that their documents, files and archives are interoperable. This to some extent is happening with H.264/ASP, or rather AVI vs MP4/MKV. AVI for many years has had "good enough" status, so much so that people simply developed hacks to squeeze every last bit of life out of it. Joe Average does not understand encoding, nor the hacks or reasonings; and so expects every encode to be stored in AVI; after all it's "good enough" (and we've seen this weak argument a number of times). Having said that, there are a fair number of encoders who do not know this themselves (as well as personal favourites like the mod 16 rule (how we love to nitpick)), which also adds to AVI's continued existence as a distribution container for modern codecs. Well in a similar vein, and jfs has already touched on this; in the same way that formats like JPEG, MP3 and AVI gain good enough status (or perhaps monopoly would be a good word), the "average fansub quality" (if you can imagine such a thing) is also "good enough". Quite rightly, people will say that they do not see the point of switching to H.264 if it does not offer any outstanding gains, but this is another problem entirely. You see here we come up against diminishing returns. Current fansub quality is considered good enough, because all things considered, the video retains a good to high quality, and compression artifacts tend to be minimal. Now if you are retaining most of the quality with a Q2 XviD, you don't make much of an appreciable gain with a Q1 XviD because of diminishing returns, and also because of the fact that the higher the quantizer, the more detail is retained (more unique values/less redundancy to exploit when entropy coding), which amounts to more high frequencies which bloat the filesize; also the eye is not so sensitive to these high frequencies, so it's generally wasted detail/bitrate. This is the reasoning behind encoders capping at Q2, because anything over Q2 is considered wastage for distribution (unless for some reason it's warranted). Now hold that thought. Let's keep this simple for examples sake and say that a 233MB XviD encode turns out at Q2. Most people would agree that encoding the XviD over 233MB would be wastage, a) because it's a fair filesize already, and b) because it's already at Q2, retaining most detail. Now encode the same file with x264, at the same filesize. With x264 being more efficient (or simply H.264 vs ASP in general), it is possible to better XviD's quality at the same filesize, meaning that lower quantizers will be used. Bearing in mind that H.264 at Q18 is roughly equal to ASP at Q2; the 233MB H.264 may end up at Q12 (unlikely it would be that low, but this is for simplicity). Q12 H.264 is about equal to Q1 in ASP, so now we have a 233MB H.264 with equivalent quality to a Q1 XviD. Obviously the quality is better, but due to diminishing returns and so on, it's perhaps not notably so to everyone. What I am getting at is that if your XviD encodes are of good quality and in the region of Q2 at a said filesize, and you encode a H.264 version at the same filesize, the gains you make in the H.264 encode will be minimal due to diminishing returns, it's akin to encoding a Q2 XviD and a Q1 XviD. In this situation where there is no outstanding difference in quality, I will tend to encode H.264 at Q18 and ASP at Q2 (give or take), and let the filesize be the advantage of the H.264 encode. While naturally I am always happy for better quality, if people are saying your XviD encodes are good enough, then to me it seems reasonable to encode the H.264 at the same or slightly better quality, but have the filesize difference, rather than keep same filesizes for quality that few will appreciate (especially so with TV rips, for DVD sources I can appreciate it though). Quote:
Quote:
__________________
|
|||
2007-02-28, 18:15 | Link #739 |
Member
Join Date: Jun 2003
Location: somewhere far beyond
|
As you already said, raw quality is the limiting factor, and if one encodes an animax or at-x raw with asp or avc doesn't matter because the outcome will look like shit regardless.
Also, there is no editing application comparable to vdub for avc (and the new formats like mp4&mkv in general), something at least I miss. Yeah, I know, dshowshource, but keyframe info is lost and in general it's way more hassle than just throwing files at vdub. CU, lamer_de
__________________
|
2007-02-28, 21:32 | Link #740 | |
Senior Member
Join Date: May 2006
|
Quote:
I'm not sure if there are other groups that follow this, since I don't usually pay attention to file sizes. My view on the matter, however, is that I would much prefer to be able to take away from an episode that would gain "invisible" quality from using a standard file size and offering the freed amount to an episode that could really benefit from a larger-than-standard file size. |
|
Thread Tools | |
|
|