¶What makes a computer slow
I've become increasingly frustrated with the speed of computers lately, or rather, lack thereof. After thinking about it, I came up with three reasons why I think computers have gotten slow. It's a bit long. Should I call this a rant? Oh well, let's just start with the first one:
#3: Multithreading
Multithreading, overall, has been good for computer usability. I'm glad that I no longer have to choose to run either my editor or my program, or shell out from my editor and remember that I've done so (ah, BRIEF). While multithreading on the CPU is good, though, it isn't so good for the hard disk. In fact, it makes hard disk performance suck, because seeks are expensive and even a moderate amount of seeking causes throughput to plummet. The problem is that when you add multithreading to the mix, you have multiple threads submitting disk requests, which then causes the disk to thrash all over the place. This is especially fun when you boot your computer and all of the garbage that every program has placed in the startup folder decides to load at the same time. It's gotten worse with faster CPUs, because now programmers think they can speed things up by doing tasks in the "background," only to increase the amount of contention on the one poor disk, which ends up lowering overall throughput. (*cough*Intellisense*cough*)
My fondest memory of the effect of multithreading on disk performance was from my Amiga days, where attempting to read two files at the same time would cause the floppy disk drive to seek for every single sector read. As you can imagine, that was very loud and very slow. (Gronk, gronk, gronk, gronk, gronk....) Newer operating systems do buffering and read-ahead, but you can still see some pretty huge performance drops in VirtualDub if you are doing a high bandwidth disk operation and have something else hitting the disk at the same time. Supposedly this might be better in Vista, as the Vista I/O manager doesn't need to split I/Os into 64K chunks, but I haven't tested that.
#2: Virtual memory (paging)
Ah, virtual memory. Let's use disk space to emulate extra memory... oh wait, the disk is at least three orders of magnitude slower. Ouch.
Contrary to popular opinion, virtual memory can help performance. In a good operating system, any unused memory is used for disk cache, which means that enabling virtual memory can improve performance by allowing unused data to be swapped out and letting the disk cache grow. In the days of Windows 95 there was a big difference between having a 500K cache and a 2MB cache... or no cache at all. Nowadays, it feels like this is very counterproductive, because Windows will frequently swap out applications to grow the disk cache from 800MB to 1GB, which barely increases cache hit rate at all, and just incurs huge delays when application data has to be swapped back in. It's especially silly to have every single application on your system incrementally swapped out just because you did a dir /s/b/a-d c:.
In the old Win9x days, you could set a system.ini parameter to clamp the size of the disk cache to fix this problem; with Windows NT platforms, it seems you have to pay $30 for some dorky program that calls undocumented APIs periodically. I wouldn't laugh, Linux folks -- I've heard that your VM is scarcely better than Windows, and the VMs of both OSes are trounced by the one in FreeBSD. Of course, I can count the number of people I know that run FreeBSD on one hand.
For better or worse, virtual memory has made out-of-memory problems almost a non-issue. Sure, you can still hit it if you exhaust address space or get absolutely outrageous with your memory requests, and the system will start running reaaaalllllyyyy slooooowwwww when you overcommit by a lot, but usually, the user has ample warning as the system gradually slows down. Virtual memory has also let software vendors be a bit loose with their system requirements -- you're a moron if you think a program will run well with 256MB when that's what it says on the box -- but at least the program will work, somewhat, until you can get more RAM.
That isn't the worst problem, though. Care to guess what my #1 is?
#1: Garbage collection
Time for my real pet peeve, garbage collection -- which has been introduced to Windows applications in spades by the .NET Framework.
What is garbage collection? It's an implicit memory management mechanism, versus the usual explicit mode. That is, instead of a program directly saying that it is done with memory, the system detects that memory is unused when the program has no more references to it. The primary advantages of using GC are that the program doesn't have to manually manage memory, which involves both CPU and programmer overhead, and true memory leaks are impossible. Memory heap related problems, such as leaks, double frees, etc. are responsible for a lot of program failures and can be a huge headache when programming, thus the interest in avoiding the problem entirely.
The classic argument against GC is that it can eat a lot of CPU time doing its sweeps, and it does so disruptively in spurts. Sure enough, in order programs with GC, you could see them noticeably halt every once in a while while the GC did its work. Yet, even though I'm a diehard I-love-performance, native-code-all-the-way kind of guy, I think this problem is mostly a non-issue at this point. For one thing, it's been a while since I've seen a program with pauses due to periodic GC CPU usage being a problem, and it seems that in modern Java and C# environments it's easy to get decent GC performance as long as you aren't unnaturally stupid with your allocation behavior. Heck, most old BASIC interpreters used it, and if garbage collection worked quickly enough for a 1MHz 6502 with 64K of RAM, it should work just fine on modern systems. Sure, you wouldn't want to use garbage collection in a program with a real-time requirements, but you're better off avoiding any dynamic memory allocation in that case.
No, the two problems are memory usage and locality.
Programs that use GC as their main memory allocation strategy consume more memory. Yeah, a good compacting heap can get better packing than a malloc heap, but GC-based programs generally allocate more memory than non-GC ones so that the GC doesn't have to run all the time, and in general I see GC programs take a lot more memory than non-GC ones. This is part of the reason I hate the new trend of writing video card control programs in the .NET Framework. I swore up and down that I would never buy another video card from a particular vendor once I saw their awful new .NET-based control panel that takes more than 50MB of memory, and of course the other major IHV went and did the same thing so I'm now screwed both ways. Repeat with dozens of other applications that have gone .NET, and it starts to add up very quickly.
You might say: why does it matter, since modern computers have so much memory? The memory would go unused otherwise, so you want apps to use it all up, right? Absolutely not. First, having a dozen programs that all claim extra memory adds up really fast, but as I've said above, the disk cache would use that memory otherwise. (In fact, in a way, the disk cache itself uses garbage collection.) If five applications each eat up an extra 100MB, that's half a gig that isn't available for disk caching. When you're working with big datasets, like editing a big sound file or linking a program in Visual Studio, that smaller cache can become noticeable.
The real problem, though, is access locality. I'm not talking about something like low-level cache behavior here -- I don't think explicit memory management has an advantage here, and GC or not, that's best taken care of by preallocating your data in clumps and avoiding allocation in your inner loop. I'm talking about the well-known problem that GCs don't interact well with paging because a full garbage collection pass requires access to the entire memory heap. It's this, I claim, that is currently ushering in the new Era of Slow(tm). Let's say you have a control panel icon in the corner of the taskbar, say, with a red icon that has three letters on it... say, an A, a T, and let's throw in an i for good measure. You haven't used it in a while and it doesn't do anything while idle, so the OS has happily paged most of it out to disk. Now it becomes active, and the garbage collector decides to do a gen.2 garbage collection sweep... which causes every single page in that application's heap to swap back in. The result is an agonizingly long wait as the hard disk goes nuts bringing thousands and thousands of 4K pages back in one at a time. Even better, if the reason your system started swapping is because you're already low on memory, this impressive amount of swap traffic causes even more swapping in other programs, grinding everything to a complete halt for as long as several minutes. Argh!
Let's be honest: garbage collection is here to stay. It's quite powerful for certain data structures, most notably string heaps, and it has undeniable benefits in other areas such as sandboxed execution environments and concurrent programming. What I think aggrevates the problems, though, are languages and programming environments that you insist on putting everything in the GC heap. It is possible to do garbage collection in C++ even without runtime help, and it's extremely valuable for caches. A compacting garbage collector like the one that the .NET Framework uses, though, has the irritating behavior that it wants to see everything so it can track and relocate pointers. The result is that it's so painful to deal with external memory that you end up with lameness like keeping 1MB images in GCed arrays. Add to this languages like C# that have very poor support for explicit memory management (doing manual reference counting without C++ style smart pointers sucks), and you get tons of programs that have much poorer memory behavior than they should. It seems that Java fares a little better -- in particular, Azureus felt much snappier than I had expected for a Java app -- but I'm sure there are counterexamples.
The usual argument for GC is that it relieves the programmer of the burden of managing memory. That really isn't true, and I think the current trend of relying on the GC to handle everything costs a bit too much performance-wise. Even with garbage collection, it's still a good idea to carefully manage object lifetimes and minimize allocations. I try to allocate as little memory as possible in VirtualDub's inner render loop, and I would do the same even if I switched to garbage collection -- not that I have any intention of doing so, ever. It's OK to give up some performance for rapid development and robustness, but I frequently see new applications go well over 100ms and into the seconds range for non-interactivity, and I believe that garbage collection is largely responsible for it.