Current version

v1.10.4 (stable)


Main page
Archived news
Plugin SDK
Knowledge base
Contact info
Other projects


Blog Archive

Watch out for number bases in Visual C++ visualizers

I just got burnt by an old bug -- err, feature -- in the Visual C++ debugger.

In the VC6 days, there was a nasty "feature" where the debugger evaluated unprefixed integers in expressions used the current number base setting. This meant that array[10] in the Watch window showed element 10 if you were in decimal mode and element 16 if you were in hexadecimal mode. You had to use the nonstandard "0n" prefix to force decimal, i.e. array[0n10]. Thankfully, this has been fixed and in the last few versions unprefixed numbers work as expected.

However, it turns out that this problem still occurs in autoexp.dat visualizers. My deque visualizer looks like this:

            #if ($e.m.mapEnd == $e.m.mapStart)

            expr: $c.m.mapStart[($i+$c.mTails.startIndex) >> 5]->data[($i+$c.mTails.startIndex) & 31],
            size: ($c.m.mapEnd-$c.m.mapStart-1)*32+$c.mTails.endIndex+1-$c.mTails.startIndex

vdfastdeque, like most STL deque implementations, stores elements in a sequence of fixed-size blocks with some portions of the first and last blocks possibly unused. Well, it took me a while to figure out that the elements I was seeing were bogus, but only when hexadecimal mode was enabled. The problem is the "& 31" in the array expr: field, which was turning into "& 0x31" in hex mode... argh!!

As expected, changing the visualizer to use 0n31 or 0x1f fixes the problem. I did some experimentation, and this only seems to affect cases where expressions are evaluated for display. In particular, it doesn't affect the size: field of #array, and probably not any conditions or values used by #if or #switch, which is why none of Microsoft's visualizers show this problem. The problem also reproduces in VC2010 Express, so it hasn't been fixed yet. That's not too surprising, since very little has changed in the visualizer engine except a couple of small bug fixes.

Anyway, today's conclusion: make sure you test your visualizers in hex mode, and prefix constants to avoid such problems.


This blog was originally open for comments when this entry was first posted, but was later closed and then removed due to spam and after a migration away from the original blog software. Unfortunately, it would have been a lot of work to reformat the comments to republish them. The author thanks everyone who posted comments and added to the discussion.