§ ¶File extension madness
Ever hit the problem in Windows where you tell a program you want it to handle some file types, and only some of them "take"?
I've got a pretty good idea now why this happens, and it's a mess.
The MSDN Library has a section on how to associate file types in code. There are some omissions -- I knew what the positive and negative numbers meant on icon references, although that doesn't actually seem to be documented -- but in all it's actually not bad. If you follow the documentation you can indeed add a UI to your program to associate file types, and if you spend another hour in the Vista UAC section you can even get your program to properly relaunch itself as elevated as needed. Then you send it out to your users, and you find out it inexplicably doesn't work on some users' systems.
Well, long story short, the culprit turns out to be yet another overlay on the file type mechanism located at HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts. This is a bit odd, because there is already a user-local overlay over the main class database at HKCU\Software\Classes. Within the key for each extension here there can be a subkey called UserChoice, which is created when the user specifically overrides the current association with the Open With... option and has the "always use this type" checked. This then overrides the ProgID selected through in HKCR. Frustratingly, I wasn't able to find any documentation on this in MSDN, but it wasn't until I tried to delete the subkey for testing in Registry Editor and got an Access Denied error that I had a clue why it wasn't documented. Let's take a look at the security settings:
A Deny entry for the current user, for only Special permissions? That's strange. What exactly are they?
Mmmhmm, the shell explicitly set a Deny entry for the Set Value permission on the key. That makes this Microsoft's latest salvo in the War Against the Coolest Most Amazing Program Ever Written, especially since bypassing this requires mucking around in security APIs that are mazes of SIDs and ACEs. While I appreciate the intent -- having had to manually clean up the mess from programs that thought it funny to claim .ZIP and .PDB as media types -- I'm not sure that the resultant game of Open With whack-a-mole that results is any better, where first I have to manually switch AVI, then MPG, then MP4, and every other extension still bound to the ugly-as-sin Windows Media Player. It would be nice if the file type documentation actually mentioned this user override too, even if it didn't include storage details, because this appears to make the documented HKCR method useless for any sort of custom association UI.
Digging around the MSDN links a bit reveals an IApplicationAssociationRegistration interface added with Vista which might work, but I haven't tried that yet. Annoyingly, it requires you to register your application in HKLM, which seems backward given the moves toward storing file associations per-user. That it offers a programmatic interface to override associations also doesn't make me hopeful since that also goes against the bunker Microsoft's been trying to build around the file type settings.
I'm struggling with this exact issue. I've added a shell verb in HKCU\Software\Classes, but as soon as the user chooses a default application through "Open With...", the verb disappears for good, due to the new association being created under HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.ext\UserChoice.
larsch - 25 10 11 - 01:16
Yeah, it's part of the Default Programs associations, if I recall correctly. One of the several file association rough patches solved by Default Programs Editor (google it). For investigation, you can export your changes in that app to a .reg file and then open it in a text editor to inspect what changes it's making.
factor - 25 10 11 - 01:24
The idea is that file type association base data (the stuff that goes in HKLM) is part of the software installation and should be added/removed by the installer and uninstaller, while the part that goes in HKCU is managed by the shell. However another part of my belief is that software installed "portable" should never even attempt to muck around with file associations and the like, because the software "installation" might move location at any time (think USB gadget getting a new drive letter), invalidating all setup it has done.
The Vista+ default program registrations thing works really well and solves almost all the problems. It just is a shame it only works for system-installed applications since it needs to be in HKLM outside Classes, so it can't be used e.g. for software installed under AppData\Local, such as Chrome does.
Niels M - 25 10 11 - 01:43
Then again, AppData\Local is not where software should be installed.
me - 25 10 11 - 08:28
Actually, AppData\Local is where per-user installations are supposed to occur. Windows Installer defaults to subfolders beneath that folder when per-user installation is selected:
Perhaps not the most intuitive place, but it's where they occur nevertheless. I think ClickOnce does similar.
Associating file types to a program that has portable-style deployment may not make sense at first, but it's handy to be able to drop an .exe in any location, even if it has to drop anchor once associated. It's especially handy if you're doing this for a program that's being developed internally -- the last thing you want to do is have to go through a full install process every time you get a new build.
The Vista interfaces do actually look pretty decent. One use case they don't appear to cover is the ability to vary the nature of the association based on options, such as whether you want the target opened in a new window or in a reused window. This isn't possible per-user if the extension-to-ProgID mappings have to be placed in HKLM.
Phaeron - 25 10 11 - 15:35
You should report back a few of those things to MSDN o.o
DJT-Rex - 25 10 11 - 19:52
What I do on freshly installed Windows is import this into registry:
Windows Registry Editor Version 5.00
Igor Levicki (link) - 26 10 11 - 06:50
Why do you remove all the keys?
Kelden - 26 10 11 - 06:57
Whelp, I actually tried using the Vista APIs and discovered another problem. There doesn't appear to be any way to clear an association besides the extremely destructive ClearUserAssociations(). I think the intended unregistration path is for you to delete the ProgID and for the shell to notice it dangling, but that doesn't help if your app supports multiple extensions with the same ProgID -- you can't disconnect some of the extensions. This shows up as grayed out checkboxes in the UI. Bleah.
Mr. Chen, this is why people go to the API dark side....
Phaeron - 26 10 11 - 18:32
This reminds me of a problem I've hit several times: the complete inability to associate a file extension with a program - right-click a file, choose Open with, click Browse, find the program you want to associate the file type with, click OK ... and the program you just selected doesn't appear in the list (and there's no error or any other indication that anything went wrong). Seems to happen if you uninstall a program that had a type associated and then reinstall it elsewhere (specifically happened to me when I removed TightVNC and wanted TigerVNC to handle all .vnc files instead - that took me a while to get to work).
ender - 27 10 11 - 04:24
One thing I couldn't find in MSDN is the 15 file selection limit. If you select 15 files or more in Windows Explorer, you can't open them with default program. e.g. if you have VLC associated to .avi files, and you select 15 .avi files in Windows Explorer, then right click, the only option is to open in Windows Media Player.
15 file selection limit - 28 10 11 - 01:14
@15 file selection limit: The limit documentation is @ http://msdn.microsoft.com/en-us/library/..
(Max 100 for static verbs and unlimited for shell extensions)
WndSks (link) - 01 11 11 - 07:01
Sorry for not explaining because I thought it was obvious -- I have my own set of associations (also imported through .reg file).
The keys I remove are default associations and context menu entries for Windows Media PLayer which I not only do not use but also despise. Some reasons for thar are outlined in this article:
Igor Levicki (link) - 03 11 11 - 02:47