¶Gotchas when creating a single instance application
A single instance application is one that only permits one instance of itself to run at a time. One reason to do this is that the program requires exclusive access to a shared resource, such as a large persistent database within the user profile. Another reason is for better UI: perhaps for an image viewer you want the same window to be reused by default instead of popping up new windows each time that the user needs to close.
Depending on the specific requirements, there are a couple of ways to enforce a single instance on Windows, such as named mutexes and exclusive file locks. In addition to enforcing the restriction, though, you'll usually want to pass the command-line parameters over as well so that the existing instance can respond to the new request. The first inclination is to just marshal over the command-line parameters via some RPC mechanism. A cursory search for sample code shows this to be a common practice. This works great, until you get a command line like this:
awesomeness.exe file.awe
The problem here is that the passed-in path is relative. Since most Win32 GUI programs will change their current directory whenever the common file dialog is used, it's likely that the existing and new processes will have different current working directories and thus the open operation in the existing process will fail. Sending over the current directory solves this problem, but this is the case I find is often forgotten:
awesomeness.exe d:file.awe
This is a drive-relative path, which causes the same problem. These are useful if you're a command-line junkie like me. The tricky part is that there aren't explicit APIs in Win32 to access the current drive paths, even though Win32 itself does the resolution. It turns out they're hidden environment variables of the form =A:, =B:, etc. Send those over too, and now the paths work. Another way to solve this would be to expand all of the paths to full paths in the second instance, but that's a bit more command-line parsing in the cross-launch path than I like.
There is a third case to watch out for, which is when the existing instance is running under a different user account within the same window session. You can get into this situation via the "runas" command or the "run as a different user" functionality. This bites me in the butt occasionally with Firefox where some program I have running under one user account will launch a web link and it'll load into the existing Firefox instance under a different user profile than it would have otherwise. Detecting this situation involves a little bit of detective work with getting the process ID of the window, opening the process, and checking the user information on the process token.
The final odd situation is elevated vs. non-elevated status on Windows Vista, i.e. run as administrator. This is a pretty odd situation since the process is running under the same user and mostly the same but not quite the same environment. A particular issue is that elevated and non-elevated processes see different sets of SUBST'ed drives. It's pretty clear that you wouldn't want a non-elevated launcher to hand arbitrary arguments off to an existing elevated process, but silently failing or throwing an error isn't a good user experience either. Handling of this case likely needs to be tailored depending on the reason for the single-instance restriction.