¶The whole reason of FOURCC codes is to prevent this mess

I really hate it when people overload four-character (FOURCC) codes for video formats they weren't intended to handle. Why?

Because it creates a mess that the user has to clean up manually.

The whole point of FOURCC codes and format structures is to unambiguously identify the format and color space of image data. When you overload an existing FOURCC with a new meaning, it creates ambiguity that causes conversion errors throughout a video processing pipeline that can be hard to track down. One common transgression is writing full range YCbCr data into the YUY2 FOURCC instead of 16-235 luma range. This creates all sorts of situations where video ends up either 14% higher or lower in contrast because two modules disagree on what range is being used, and they can't autonegotiate because the same code has been overloaded. Ditto for Rec. 601 vs. Rec. 709 coefficients, or interlaced vs. noninterlaced.

One solution that is often suggested is to make it a user choice. Well, that's one way to "solve" the problem, but it propagates the mess, and it also assumes that the user knows the answer to the question. Do your users whether the video codec you are using uses full-range luma or not? Does the vendor who makes the video codec document that? Probably not. It gets worse, though, when you consider that there may be hidden conversions in the pipeline. You can argue that hidden conversions are bad, and in general it's better to avoid them for performance and quality reasons. However, they're sometimes unavoidable, particularly for display. What would you do if you saw this dialog:

...followed by more for the video compressor and for the video driver. Heck, at this point, why bother having FOURCCs at all? Just make the user enter in the color space and chroma subsampling information too.

I've also heard the suggestion to make the determination based on whether the frame size is SD or HD. That's not really any better. So now you can't resize video temporarily across the SD/HD boundary without incurring a color conversion... yuck.

An additional reason I've heard to put in options is to recover blacker than black (BTB) and whiter than white (WTW) areas. Well, yes, you can do that via changing the color space, but that's not really what you're trying to do -- you need some sort of conversion to map those values in range, but the full range YCbCr color space is unlikely to be the one you want, and by changing the color space you're also whacking chroma as well as luma. What you really need is a custom color conversion matrix or at least to remap the luma range. Otherwise, you're likely reducing contrast more than necessary or introducing slight color shifts into your video.

At this point, I've resigned myself to having to eventually add some of these options to VirtualDub, because I see far too many cases where people have broken color conversions in their pipeline and they need to be able to compensate for codecs and players that use the wrong conversion formula. However, I fully intend to put a big "THIS IS NON-STANDARD" flag on the options, because I do not want to further the idea that this is the way that FOURCCs are supposed to work. Doing this is a big change to the blitter and display libraries, too, so I have no idea when this would be implemented.

Comments

This blog was originally open for comments when this entry was first posted, but was later closed and then removed due to spam and after a migration away from the original blog software. Unfortunately, it would have been a lot of work to reformat the comments to republish them. The author thanks everyone who posted comments and added to the discussion.