Guidelines for ColorMatrix usage with .ts sources?

[natsume_g]

I've already asked this on darkhold, but in my current situation I have connectivity problems, so I was disconnected from irc before I could wait for a response. I figured here would be another good place to ask.

Are there any guidelines for using colormatrix() on mpeg-2 transport stream sources that use Rec.709 coefficients? I've read that the default coefficients for mpeg-4 AVC (my target) are Rec.709 for HD and Rec.601 for SD, but is this accurate? If it is, my assumption is that if you go from HD .ts -> HD h264 you should not use colormatrix, but if you go from HD .ts -> SD h264 you should, but I want to make sure that I'm understanding this correctly.

Thanks in advance!

[TheFluff]

It's a valid question but there isn't a clear answer. You not only need to know the source coefficients, you also need to know what coefficients the renderer side will use to convert the video to RGB (for display on the monitor). This is different between renderers; most (including the by far most common one, the overlay mixer) use Rec.601; some (Haali's) can be configured to use either Rec.709 or Rec.601. Haali's renderer can also use "auto" which means that it'll make the somewhat interesting assumption that all video wider than 1024 pixels uses Rec.709.

The entire thing is complicated further by the fact that x264 has a metadata setting that tells the decoder which coefficients are used; by default it's undefined but you can explicitly set it to whatever you want. The question is if any decoders read this flag, and if they do, what they do with the information. I'm pretty sure there aren't currently any decoders that actually do anything with this information.

To sum it up, your safest option is probably to use ColorMatrix to convert to Rec.601. That way it will usually have the correct colors with the most common renderers; people who use Haali's renderer or other renderers that can be configured to use other coefficients can probably be assumed to have at least some clue about what they are doing and can change the settings if it looks wrong.

[checkers]

And don't forget if people manually convert to RGB with the decoder there's another point where you can change the matrix in use.

[cyberbeing]

Originally Posted by TheFluff

This is different between renderers; most (including the by far most common one, the overlay mixer) use Rec.601; some (Haali's) can be configured to use either Rec.709 or Rec.601. Haali's renderer can also use "auto" which means that it'll make the somewhat interesting assumption that all video wider than 1024 pixels uses Rec.709.

I don't think this is accurate. To my knowledge both ATI and nVidia (I know nVidia does for sure) hardware, will use Rec.709 on VMR9/EVR and Overlay Mixer with the same or similar behavior to Haali's auto setting (something like >1024 width). I've tested it before on my nVidia card, and this has been the case for at least the past couple years. With that said, stick with Rec.601 for SD and Rec.709 for HD because that is what all the common renderers will use for those resolutions.

[comatose]

I use BT.709 for HD and BT.601 for SD, then flag the stream when encoding with x264 using --colormatrix.

It's the best you can do really - just have to pray that the converters that *don't* decide which colormatrix it is based on resolution will check the flag.

[Nicholi]

Originally Posted by cyberbeing

I don't think this is accurate. To my knowledge both ATI and nVidia (I know nVidia does for sure) hardware, will use Rec.709 on VMR9/EVR and Overlay Mixer with the same or similar behavior to Haali's auto setting (something like >1024 width). I've tested it before on my nVidia card, and this has been the case for at least the past couple years. With that said, stick with Rec.601 for SD and Rec.709 for HD because that is what all the common renderers will use for those resolutions.

I know my nVidia card certainly does not decode to Rec.709 for HD material. It uses Rec.601 coefficients all around, no matter the resolution. As I recall it's been listed as an "issue" in their driver release notes since forever (still there now).

The RGB->YV12 conversion is controlled by the card's drivers (see software not hardware). The renderer does not have much choice in the matter, no matter which one you use (unless you use Haali's renderer which lets you specify, but we are talking about automagic things). The renderer is not doing the conversion, it is just feeding the video to the card with hopefully the instructions that it can follow.

Which makes the problem down to the uncontrollable element of every video card maker's drivers. Which I'll bet the majority is still using Rec.601 (for all resolutions), as they have been for years. Which nVidia drivers are using Rec.709 for HD resolutions and which aren't? Same for ATI and every other brand name. Who the fuck knows. If you are going to make assumptions you should probably go with what has been the "standard" for years. Which of course just makes things fucked.

Also it is not that all HD media is meant to be decoded with Rec.709 coefficients, it's that Rec.709 is used by Blu-ray, HD DVD (lol), and HDTV standards. Specific applications of HD media. And we all know PCs do alot moar than play those formats. So it is not as though it should be obvious, even to people writing drivers, that all things with a certain resolution should suddenly be decoded with Rec.709. For now it probably would be more correct to assume (if you are making assumptions) most users will be decoding with Rec.601. Whether the people writing video drivers start following a new standard that says all videos with X or greater vertical/horizontal lines decodes with Rec.709 or not is yet to be seen. I can see it going either way really. Or going no way and just continuing to be the muddied confusing thing it is now ;-;.

Originally Posted by comatose

I use BT.709 for HD and BT.601 for SD, then flag the stream when encoding with x264 using --colormatrix.

It's the best you can do really - just have to pray that the converters that *don't* decide which colormatrix it is based on resolution will check the flag.

Well you might think it's the best thing that can be done but since that option isn't implemented in any of the majorly used H.264 decoders (neither ffdshow or CoreAVC) it's pretty much hopeless thinking. As the x264 docs even say it is an extremely obscure option and the likelihood of it even being implemented by the decoders is low. CoreAVC just works off the automatic all videos over 1024 width are Rec.709, and ffdshow is completely manual (the default option being Rec.601).

The option wouldn't matter much anyways since almost everyone is outputting YV12 video (not RGB) with their decoders. Meaning again the drivers are what is handling the conversion. The decoder doesn't even enter the equation, even if it did have the option implemented.

The only real "best" thing you can do to assure everyone is correctly playing your releases is to tell them to force output to RGB in the decoder (and set the coefficients manually if required). Since currently it seems you will have no idea whether their drivers are automatically using Rec.709 for HD resolutions or not. But then that extra conversion just makes things ugh slower, and you are already dealing with HD resolution H.264.

Pretty much now it's a crapshoot. Though probably best to keep HD things Rec.709, even if it means the majority of people aren't watching it correctly.