¶Direct3D may not show custom video modes
I spent some time today trying to get some Direct3D 9 code to switch the display into 50Hz refresh so it could display PAL video. Displaying PAL video on a 60Hz display doesn't look very good because of the difference in frame rates -- you get a beat at the difference between the rates, 10Hz in this case -- and so it's better to change the display mode to match. Problem is, a 50Hz mode wasn't showing up in the mode list for my monitor, so I opened the NVIDIA control panel and added a custom 50Hz mode. The test ran fine and the monitor happily switched into a PAL-compatible refresh rate.
Then I tried the Direct3D 9 code, and it refused to do the mode switch.
Strangely, the old ChangeDisplaySettings() call had no problem selecting the 50Hz mode and the control panel showed 50Hz available, but a check with the DirectX Caps Viewer revealed that the mode wasn't showing up in the Direct3D APIs. Checking the docs for the IDirect3D9::CreateDevice() method revealed this ominous comment:
An unsupported refresh rate will default to the closest supported refresh rate below it. For example, if the application specifies 63 hertz, 60 hertz will be used. There are no supported refresh rates below 57 hertz.
It would be disappointing if the Direct3D 9 API was ignorant of a few continents worth of video standards, but fortunately this doesn't seem to be true at least on Windows 7. Some 56Hz modes were showing up and I could switch to those just fine, so that statement appears to be wrong.
A bit of digging with WinDbg -- okay, a frustrating amount of digging -- showed that on Windows 7, d3d9.dll uses the EnumDisplaySettings() API to enumerate available video modes. This function unfortunately only returns modes that are determined to be compatible with the current monitor. Even worse, if you happen to have your desktop set to an "unsupported" resolution, Direct3D grudgingly temporarily returns the mode in its list, which can lead to confusion. I pulled the EDID stream for the monitor and it showed that the monitor was only saying that it supported refresh rates in the 56-75Hz range, even though it actually supports both 50Hz and 85Hz as well. EnumDisplaySettingsEx() with the EDS_RAWMODE flag did return the 50Hz mode, but that wouldn't help.
Plan B was to try to use the D3DPRESENTFLAG_UNPRUNEDMODE flag, which supposedly allows access to raw modes. It requires using Direct3D 9Ex instead of Direct3D 9, but fortunately my code worked both ways. Unfortunately, there still isn't a way to enumerate the hidden modes, and setting that flag didn't seem to do anything. More digging revealed that when 9Ex is in use, d3d9.dll uses D3DKMTGetDisplayModeList() to enumerate the modes instead. That function does return all modes, but farther down the line a function called d3d9!BuildModeTableLH() has a hardcoded check for the D3DKMDT_DISPLAYMODE_FLAGS::ValidatedAgainstMonitorCaps bit and drops any modes that are missing it:
66ff9fcf 8b9084941967 mov edx,dword ptr d3d9!g_ModeInfo+0x4 (67199484)[eax] 66ff9fd5 8b4508 mov eax,dword ptr [ebp+8] 66ff9fd8 f644022401 test byte ptr [edx+eax+24h],1 66ff9fdd 0f848a000000 je d3d9!BuildModeTableLH+0xa1 (66ffa06d) 66ff9fe3 8b0b mov ecx,dword ptr [ebx] 66ff9fe5 8b550c mov edx,dword ptr [ebp+0Ch] 66ff9fe8 8b02 mov eax,dword ptr [edx] 66ff9fea 57 push edi 66ff9feb 51 push ecx 66ff9fec 50 push eax 66ff9fed 56 push esi 66ff9fee e88c000000 call d3d9!AddModeEntry (66ffa07f)
This prevents 9Ex applications from switching to unvalidated modes even if the UNPRUNEDMODE flag is set. I'm not actually sure what that flag does anymore; according to Google no one seems to be publicly using this flag and maybe this is why. Hacking out the check works, but that's not a shipping solution.
And no, unchecking the "hide modes that the monitor cannot display" checkbox doesn't affect any of this. I unchecked that a long time ago.
In the end, I wasn't able to find a programmatic way to cleanly enable these modes for Direct3D 9 applications. What did work is to override the EDID stream for the monitor to enlarge the vertical refresh rate range to 50-75Hz. This requires a collection of tools and generating a replacement monitor INF, so it's also not a shipping solution, but it appears to be the only choice if the desired custom mode falls outside of the EDID specs.
Appendix
While tracing through the mode enumeration logic in d3d9.dll I also discovered a weakness in its IDirect3D9::EnumAdapterModes() implementation. In Windows 7, and presumably in Vista, this function is implemented on top of the internal version of IDirect3D9Ex::EnumAdapterModesEx(), which returns an extended mode structure and has additional filtering options. The non-Ex version calls through to the -Ex version, discards modes that don't pass a default filter, and then translates the mode structure. When you realize that the API for both functions is to take a mode index, though, you might see the problem. When the non-Ex version is called requesting the Nth mode, it can't simply pass through the value N since it uses a filter, so what it does is to count from 0 on up until it finds the Nth mode that passes the filter. Unlike EnumDisplaySettings() there is no caching of the filtered mode list and therefore enumerating all modes with IDirect3D::EnumAdapterModes() is an O(N^2) operation. It's best to cache the mode list if you need it frequently.