§ ¶Watch out for number bases in Visual C++ visualizers
I just got burnt by an old bug -- err, feature -- in the Visual C++ debugger.
In the VC6 days, there was a nasty "feature" where the debugger evaluated unprefixed integers in expressions used the current number base setting. This meant that array[10] in the Watch window showed element 10 if you were in decimal mode and element 16 if you were in hexadecimal mode. You had to use the nonstandard "0n" prefix to force decimal, i.e. array[0n10]. Thankfully, this has been fixed and in the last few versions unprefixed numbers work as expected.
However, it turns out that this problem still occurs in autoexp.dat visualizers. My deque visualizer looks like this:
vdfastdeque<*>{
preview(
#(
"size=",
#if ($e.m.mapEnd == $e.m.mapStart)
(
[0]
)
#else
(
[($e.m.mapEnd-$e.m.mapStart-1)*32+$e.mTails.endIndex+1-$e.mTails.startIndex]
)
)
)
children(
#array(
expr: $c.m.mapStart[($i+$c.mTails.startIndex) >> 5]->data[($i+$c.mTails.startIndex) & 31],
size: ($c.m.mapEnd-$c.m.mapStart-1)*32+$c.mTails.endIndex+1-$c.mTails.startIndex
)
)
}
vdfastdeque, like most STL deque implementations, stores elements in a sequence of fixed-size blocks with some portions of the first and last blocks possibly unused. Well, it took me a while to figure out that the elements I was seeing were bogus, but only when hexadecimal mode was enabled. The problem is the "& 31" in the array expr: field, which was turning into "& 0x31" in hex mode... argh!!
As expected, changing the visualizer to use 0n31 or 0x1f fixes the problem. I did some experimentation, and this only seems to affect cases where expressions are evaluated for display. In particular, it doesn't affect the size: field of #array, and probably not any conditions or values used by #if or #switch, which is why none of Microsoft's visualizers show this problem. The problem also reproduces in VC2010 Express, so it hasn't been fixed yet. That's not too surprising, since very little has changed in the visualizer engine except a couple of small bug fixes.
Anyway, today's conclusion: make sure you test your visualizers in hex mode, and prefix constants to avoid such problems.
Comments
Comments posted:
I ran into a nasty problem in VS2010 which seems to corrupt binaries built on exFAT partitions:
http://www.pretentiousname.com/vs2010exf..
No big deal if it only affects exFAT (except if you happen to use that for your source like I did, and don't realise what's going on) but my worry is it might affect other filesystems, just less often. So I'm holding off on VS2010 for a bit longer until this is better understood.
I also ran into some problems due to STL changes, but the changes do make sense so I can't complain. The only one which worries me, because it's caught at runtime not compile time, is that non-const std::string will now throw if something calls operator[] with the index of the null. Makes sense (nothing should be able to modify that null), but can also trip up code which was just testing the character and had no intention of changing it. (Personally, I'd only do that via the buffer returned from c_str(), but I've already run into one library which triggered the issue.)
Hmm, it'd be nice if C++ could choose const versions of methods/operators based on what will be done to their results, rather than what type of object/reference they were called on. That's probably quite difficult to do, though. :)
Leo Davidson (link) - 16 04 10 - 22:33
Oh, just noticed that before I'd posted my comment you had already seen my Connect bug report and also confirmed it.
Thanks for taking the time to do that!
Leo Davidson (link) - 16 04 10 - 22:39
The manifest tool (mt.exe) is one of the flakier parts of the VC++ build chain; in previous versions it used to crash on partially created EXE files, as would happen if you interrupted a link. I don't know what it's doing that's filesystem sensitive, but I think at one point it had a "FAT32 Workaround" switch. I have a lot more confidence in the filesystem team in the Windows group than I do in whoever wrote the manifest tool.
Phaeron - 17 04 10 - 07:08
"I have a lot more confidence in the filesystem team in the Windows group than I do in whoever wrote the manifest tool."
Aye. MT.exe seems a little abandoned to me. Even in the Windows 7 SDK it's still a 2005 version that issues a warning when you add the Windows 7 / Vista compatibility GUIDs to the manifests. :( (I found a forum post from someone saying there is a 2006 version in a Vista SDK which doesn't have the problem, but I'm damned if I can find it.)
The ability to set those compatibility GUIDs still seems to be missing from the VS2010 IDE, too, unless it's triggered in an unusual way or I'm just blind. I was expecting not to have to create & merge manifests by hand anymore but seems we still need to. (The "DPI aware" manifest flag is exposed in the IDE now, at least.)
Leo Davidson (link) - 17 04 10 - 10:15
Comment form
Please keep comments on-topic for this entry. If you have unrelated comments about VirtualDub, the forum is a better place to post them.
