§ ¶Working around display brain damage in Windows Vista
I've been struggling with video display issues in VirtualDub under Windows Vista for a while now, as some of you may know. I hit a couple of snags during the beta, one of which was due to a DirectDraw implementation issue in the OS that was fixed in RTM. 1.6.17 works decently well in Vista, fortunately. However, as I've optimized and reworked the display code in 1.7.x, I'm finding that I'm hitting a lot of weird issues in Window Vista again that I wasn't seeing in Windows XP. I spent part of last weekend fighting these again in another fit of frustration over things not displaying when they should.
When I see the exact same issues on two machines running Vista, one with an NVIDIA card and one with an ATI card, I'm inclined to believe it's Microsoft's fault.
The first problem, which I've mentioned before, has to do with DirectDraw hardware video overlays -- these are essentially secondary displays that are composited on top of the main one in the video scanout hardware itself. Yeah, yeah, Microsoft's been saying that video overlays are outdated... but they're the only widely available way to do hardware YCbCr color conversion for commonly used formats without requiring 3D pixel shaders of some sort. You'd be hard pressed to find a system out there with a resolution greater than 800x600 that doesn't support a YUY2 overlay. Well, the problem is that Vista will happily let you create a hardware overlay surface, populate it, and show it -- without actually displaying anything. Your program thinks its happily displaying video, when the user is actually seeing green, magenta, or whatever you use for your colorkey. Lame. I worked around this in VirtualDub 1.7 by calling DwmIsCompositionEnabled() if it is available, and forcibly disabling overlays if the DWM is compositing.
The second problem is more insidious. For various reasons, I'm moving the display code to a separate thread in 1.7.2, and this is exposing a lot of weird threading issues in Windows, like the HTTRANSPARENT issue I mentioned earlier. Well, another problem I hit is that the DWM doesn't seem to consistently update its composition tree when you have a child 3D window in another thread -- you can call Present() in Direct3D or SwapBuffers() in OpenGL, and nothing shows up. In fact, you get junk from underneath the window. I beat my head against the desk for hours trying to figure this out, and made the following conclusions:
- That the DWM eliminates flicker is BS. It can sometimes make it worse, because changes in the window hierarchy that were ordinarily atomic can now be visible. Formerly, if you deleted a child window and created a new one in its place, you were OK because the message pump didn't run in between and WM_PAINT couldn't be sent to any windows on the thread to cause flicker. Now the DWM can rebuild the composition tree in between and add flicker to a previously solid display.
- Calling GetDC() causes a Direct3D-based window to stop updating. Okay, sure, I can understand not wanting to support mixed GDI and D3D rendering at the same time on the same window, but it'd be nice to have some better sort of fallback mechanism than just ignoring all of my Present() calls -- at least show something, even if it's slow and flickery. Oh, and it's really hard to figure out if a window is visible without an HDC (I use RectVisible() for this). I ended up calling GetDCEx(hwnd, NULL, 0) on the parent window instead.
- You can call BeginPaint() safely, although I suspect you have to be really careful about what you do with the HDC.
- Doing a FillRect() every frame makes the D3D rendering show up... not that I'm about to ship that solution.
- WS_EX_COMPOSITED and WS_EX_LAYERED do squat.
The solution I finally came up with was to call SetWindowPos() with the SWP_FRAMECHANGED message after the first call to Present() or SwapBuffers(). This seems like an utterly bogus solution, and I see a frame of garbage whenever the D3D or OpenGL minidriver reinitializes, but in the absence of any better solution or any diagnostics to determine what's really going wrong, it's the best I can do. Sigh.
I think the most astonishing part to me is how Microsoft can form a movement to get applications "Vista compatible" -- when in reality what they've done is broken a lot of apps and asked the vendors to pick up the pieces. Sure, some apps were doing really broken things, but I'm just trying to use Direct3D to display video....
Sure that this ain't got something to do with Vista's all new and shiny DRM stuff?
According to Peter Gutmann's article, you might have hit the magic 520k pixel limit of "premium content"?
pb - 03 04 07 - 05:50
If there were a low limit like that on the size of a texture in D3D, then that would break a number of games that came out before vista was close enough to shipping for the game companies to send in for whatever is needed to tell vista "I'm not a DRM-breaking program! let me in!"
Although, I do think that MS went overboard on the paranoia in Vista, from what I have heard from a number of sources.
Coderjoe - 03 04 07 - 13:16
Maybe it's teh way MS forces developers to move to WPF ... and make their software as slow and bloated as Vista itself.
mors - 03 04 07 - 18:08
Let's put away the tinfoil hat for a second... the image degradation only applies when premium content is present, which certainly isn't the case. I don't even install anything on my Vista installations besides video drivers.
As for WPF (a.k.a. WTF), well, I tried using XAML just to stretch a static image onto a window, and it was unbelievable how slow and flickery the result was.
Phaeron - 04 04 07 - 01:57
I honestly think the whole windows API is a mess. It seems Microsoft has yet to escape from 1980s programing models. Why haven't they made a new, more useful API? MFC doesn't count.
TechMage89 - 04 04 07 - 10:23
Avery, have you contacted anyone at Microsoft regarding these issues? If some of these are confirmed bugs, itíd be a good idea to provide feedback to the area owners and have them push for getting appropriate fixes into SP1 (or a hotfix). Iím guessing you probably have a few contacts at MS at this point (considering the popularity of VirtualDub), but if youíre lost, send me an email and Iíll see if I can find the appropriate area owners.
Alex - 04 04 07 - 17:08
Personally I'm peeved that even though they do desktop composition in 3D now, child windows are still not allowed to be translucent... I mean WTF, you're doing everything in 3D, how come making a child window translucent is a problem???
I'm also getting weird issues if I have the main window visible, along with a secondary window (non-child) visible. Opening a child window in the main window with video playing (non-overlay obviously), doesn't show (the video remains ontop of the child window). For the child window to show, I have to minimize and then show it again.. really annoying.
Blight - 05 04 07 - 08:42
I've sent you email.
Sadly, you seem to be correct. As much as I tried, I was never able to get a translucent child window, even with both parent and child windows doing Direct3D rendering with an A8R8G8B8 backbuffer. I can get the top-level window to render translucent using WS_POPUP and DwmEnableBlurBehindWindow(), but the child window is always composited opaque. Enabling WS_EX_COMPOSITED on the child window produces even more unusual effects (it's composited with the wrong source and destination rects!). I even tried creating a window frame on the child window (WS_OVERLAPPEDWINDOW|WS_CHILD|WS_VISIBLE), but it came up opaque, Aero Basic style. Yuck!
I only need one new API: DwmGetDirect3DDevice(). :)
Phaeron - 06 04 07 - 04:12
> Personally Iím peeved that even though they do desktop composition in 3D now, child windows are still not allowed to be translucentÖ I mean WTF, youíre doing everything in 3D, how come making a child window translucent is a problem???
I know very little of how vista draws windows, but if the main window is drawn offscreen then they would need pretty complicated blending formulas (considering the most common non-premultiplied source alpha blending) to maintain the alpha channel throughout the offscreen drawings and the final blend agains the desktop and other windows at the same time.
Gabest - 09 04 07 - 08:15
Actually, compositing translucent surfaces isn't hard -- basically, you blend (rgb, 1) into the destination according to alpha. You only need the texture stages or pixel shader to output float4(tex.rgb*tex.a, tex.a), and to set the blend factors to srcfactor=1, destfactor=1-srcAlpha. Repeat until you get to the backbuffer, where you can use the standard srcfactor=srcAlpha, dstfactor=1-srcAlpha if you don't need dest alpha.
Phaeron - 09 04 07 - 23:36
Looks to me as if Microsoft is continuing their trend of forcing people to buy Microsoft software (Office 2007) by making all other software incompatible with the newest Windows versions. This is how they won and maintained their monopoly from the beginning.
SF - 13 04 07 - 21:45
Uh, again, down with the tinfoil hat. Microsoft Visual Studio 2005 SP1 doesn't work very well on Vista either, at least without the beta Vista patch....
Phaeron - 13 04 07 - 23:46
Vista works fine.... if you don't plan on using any other applications on it. It's kinda sad that MS actually consider this a release version. I don't know anyone who having actually used it would want to buy it for personal use.
You have my sympathies for having to program for it.
SonOfAdam - 14 04 07 - 21:39
Tired of MS BS? Your work would be more appreciated by the Linux community.
Linux User - 15 04 07 - 10:03
The Windows community appreciates my program just fine... and I know exactly zero people who use Linux as their main workstation OS. That is not a switch I will consider soon, I'm afraid.
Phaeron - 15 04 07 - 16:12
I have used your program for years and have admired it, but I've switched to Linux as my main OS recently and love it. Here's hoping I can find an equivalent. It's your program, and AVISynth that motivate me keep windows around at all. Thanks for all the work you continue to do to make such a quality program.
df - 16 04 07 - 12:59
Well, count one: my workstation is now Linux only. Only non-free software running on it is the Nvidia 3D driver. Too bad your software doesn't quite run under Wine yet (it can't detect the Xvid codec I installed on it).
Mitch 74 - 17 04 07 - 08:17
I for one, after 3 years with Linux, can't wait for my free copy of Solaris Express to arrive at my doorstep. Various issues, like the general immaturity of the Linux community (1 such example would be that Linux already has a lot of filesystems available, but no
, they're going to be like the guy who wanted more Lego pieces from me when I was in kindergarten, and scream 'Open Source ZFS!! Please!!!') or people calling up those Australian guys who've switched Linux distributions just to tell their IT department 'You're Retarded' and various other things.
To segue into the topic of VDub on Linux, let me tell you about this cinelerra program or whatever they advertise as a VDub replacement. As with all Linux programs, the interface is inconsistent with the rest of the environment. Even when you're running a GTK2 based WM. It's just like running a GTK2 app within a non-GTK2 WM... it looks fugly. If you do port to Linux, please don't make that mistake. Make it consistent with Qt, GTK2 or whatever. Besides, with a little bit of tweaking, I'm sure Solaris will run it well too.
randomshinichi (link) - 24 04 07 - 12:13
Please realize that Vdub runs just fine under WINE. You will have do the following to get things to work properly.
1: Disable ddraw (d3d, opengl overlay)
2: Manyually add the codecs to your system.ini ( "vidc.XVID=xvidvfw.dll" )
I have appreciated Vdub under linux for the past two years, and am more than content to run it under WINE.
Thank you for your time,
Frank Russo - 25 04 07 - 09:52
The WINE guys *still* haven't fixed the DirectDraw blit bug?? That's been in there for at least a year! I'm pretty sure I'm not the only one hitting it, too.
Phaeron - 26 04 07 - 23:47
Stefan D. appears to be working on it. I will be able to test it later this week when 0.9.36 is released. I'll make certain to update the winehq.com appdb entry with my findings. That said, I am more concerned with "works" than "works quickly" since most of my jobs run overnight. I would recommend that we continue any further discussions on the topic on the wine-users mailing list rather than hijacking this forum (Sorry AL, my bad).
Frank - 27 04 07 - 07:42
Thanks for this article. Now I know that I am not alone with the Vista & OpenGL flickering and redrawing problems.
The SetWindowPos() with SWP_FRAMECHANGED after SwapBuffers() hint helped to get over the worst problems, but I hope Microsoft fixes this, because the flickering during resizing is really ugly.
Vlasta - 10 09 07 - 15:51
>>Looks to me as if Microsoft is continuing their trend...
Not only that Avery, just look at yourself using platform specific DirectX API instead of OpenGL -- how did they manage that?
Igor (link) - 07 01 08 - 21:15
I'm also using Linux as my main OS.
I installed it maybe a month/month and a half ago, and I love it.
I've found AviDemux to be a worthy opponent to VirtualDub, but it lacks the ability to decode a simple AAC stream.
It also can't handle certain types of H.264 streams, which is a real pain in the butt.
It's still pretty good, though. I'd recommend it to anyone running that liked VirtualDub, but can't use it anymore due to The Great Incompatibility War.
Mike - 30 03 08 - 23:07