Acid800 is an test suite used to test Altirra, an emulator for the Atari line of 8-bit home computers. It is named after the famous Acid browser tests and runs a series of tests that stress corner cases in emulators but which all pass on real hardware, reporting any discrepancies in behavior. As it is designed to run unattended and on stock hardware, it is limited to testing deterministic program visible behavior, and avoids tests that are subjective or require human assessment, such as color output. The intent is that every test in the test suite can unambiguously and reliably determine whether emulator behavior is correct or incorrect.
Although the test suite is meant to test one particular emulator, it has been tweaked and designed to run on any emulator which emulates the Atari hardware to sufficient accuracy. Because many of the tests rely on the same hardware functions, a single emulation bug can result in several tests failing and the pass/fail ratio is not a good indicator of emulation quality. The test suite is also biased towards testing program-visible corner cases instead of commonly used behavior, so it is not representative of the features most commonly used in production software, nor is it a comprehensive hardware test.
If you are running the Acid800 tests and have trouble interpreting the test results, or if you are an emulator author and need help tracking down a specific issue, or even if you just have general comments, please drop me a line and I'll be glad to help.
— Avery Lee <virtualdub.org / phaeron (reverse for email address)>
The Acid800 test suite is distributed as a bootable ATR disk image. Simply mount the disk image in an emulator or through an SIO2PC/APE emulation cable to a real Atari, and boot the Atari system. It is recommended that the test be run without BASIC or any other cartridges installed, although it will run with either BASIC enabled or an 8K cartridge as long as 80-column mode is not used. Acid800 will automatically detect whether the system is PAL or NTSC.
Shortly after booting, Acid800 will wait five seconds for a request to change options. Pressing a key will bring up the options menu, from which you can fix incorrect CPU detection, disable use of 6502 undocumented (illegal) instructions, enable prompting after each test or failed test, or enable one of two 80-column display modes. The 80-column modes require a high-quality video output, no cartridge or BASIC, and optionally a Video Board XE expansion with an FX1.20 or later core.
Afterward, Acid800 will begin running the test suite. The disk image should be kept mounted during the test as tests are incrementally loaded from disk. During operation the tests will frequently have to take full control of the Atari hardware, so it is normal for the display to blank out or display test patterns during test execution. Most tests execute in a second or less and the entire suite should take no more than a couple of minutes to finish. The test suite will report and attempt to recover from any failures, although it is possible for it to crash if there is a catastrophic failure.
Acid800 contains a lot of precision code in its tests, so it requires a high degree of hardware compatibility in order to run correctly. Hardware expansions on real hardware or special options in an emulator can cause deviations from expected behavior, and this should be taken into account when interpreting test results. If you are getting unexpected failures on real hardware and haven't modified your Atari, well, that's a bad sign....
Acid800 will automatically check for and disable some tests if it sees detects a 65C02 or 65C816 CPU. However, as many of its tests are extremely timing critical and the presence of such a CPU means a non-standard accelerator, some tests may fail with such a CPU on a real system.
All of the tests are also available in standalone form in a subdirectory; these run only the test in question and then stop. Labels and listings have been provided for each test.
The Acid5200 suite is an appropriate subset of tests relocated for the 5200 SuperSystem and packed into a cartridge. It operates similarly to the main Acid800 suite. Use a 5200 controller in port 1 to control the test. The main suite is a standard 5200 32K cartridge, while the standalone tests are 8K cartridges.
The AcidSAP suite is an even stricter subset of tests for players that support the TYPE D profile of the Slight Audio Player (SAP) format. This tests the CPU, POKEY, and a small portion of ANTIC, but not GTIA. Because a SAP player only has audio output, the test suite will emit a series of pips as it executes each test, and then a small victory tune or raspberry when it has finished. The console output is available at $1000-17FF in memory.
Tests non control flow, non stack instructions. This test will fail if abs,X and abs,Y addressing modes do not wrap around the 64K address space or zp,X, zp,Y, and (zp,X) modes do not wrap in zero page.
Checks whether certain bits in the 6502's P register are handled properly. In particular, it detects whether the (B)reak bit can be changed under program control, which it should not.
Checks if basic decimal mode operations work correctly. It is not an exhaustive test, however.
Checks whether addressing modes take an expected number of clock cycles, including some cases with page crossings. This test relies on the VCOUNT register in ANTIC and will fail if VCOUNT timing is too far off.
Tests for two 6502 bugs: the famous JMP (abs) page bug, and BRK being overridden by a concurrent NMI.
Checks for correct IRQ acknowledge timing around the CLI and SEI instructions, including whether an IRQ handler can be entered with the I bit set.
Runs a series of checks for various undocumented 6502 instructions. This test will be automatically skipped if a 65C02 or 65C816 CPU is detected.
Checks whether undocumented registers in ANTIC return the correct value.
Checks for correct functionality and timing of the DLI and VBI bits in NMIST.
Checks for correct cycle and value ranges for the VCOUNT register. This test will fail if WSYNC timing is incorrect.
Checks whether ANTIC display list DMA correctly wraps at 1K boundaries and playfield DMA wraps at 4K boundaries. The playfield DMA wrapping test will fail if GTIA player-to-playfield collisions are not implemented.
Checks for correct ANTIC function when a display list is longer than 240 visible scanlines and is not manually reset by the CPU during vertical blank.
Checks for correct timing of entry into DLI handlers. This test will fail if CPU NMI acknowledge timing is off.
Checks for ANTIC response in the $D420-D4FF range.
Checks for correct player/missile DMA function, including automatic enabling of missile DMA when player DMA is enabled and per-scanline DMA in two-line mode. This test relies heavily on exact WSYNC + CPU timing and player/missile/playfield collisions.
Checks output pattern for character modes (ANTIC modes 2-7), including inversion and vertical reflection. This test relies on P/M graphics collisions.
Checks for correct positioning of DMA cycles within the scanline for various playfield modes. This test requires correct STA WSYNC timing and 9-bit RANDOM sequencing. It does not require POKEY initialization mode, however. The sub-modes break down as follows:
Mode 04-b is thus the second scanline of a narrow IR mode 4 line.
This test also checks the timing of display list DMA, but not player or missile DMA.
Checks that the CPU can be induced into ignoring an NMI from ANTIC by entering the interrupt sequence at precisely the correct cycle.
Checks for the correct value being returned by undocumented POKEY registers.
Checks that the 17-bit psuedo-random number generator circuit in POKEY resets when initialization mode is enabled and returns the correct value after a set number of cycles. This test requires correct WSYNC timing.
Checks that an IRQ occurs at the correct time when an active IRQ is enabled via the POKEY IRQEN register. This test will also fail if CPU IRQ acknowledge timing is incorrect.
Note: There is a one-cycle leeway in this test to account for temperature sensitivity in some systems.
Checks that timer IRQs can be issued from POKEY with cycle granularity when using the 1.79MHz clock. This test will fail if POKEY interrupts are being emulated with scanline granularity.
Checks if enabling two-tone mode affects IRQs issued by timers 1 and 2.
Checks if the special unlatched serial output complete IRQ (IRQEN/ST bit 3) is handled properly.
Checks that POKEY timers can be used to issue periodic IRQs.
Checks that enabling asynchronous receive mode on the POKEY serial port also correctly affects operation of timers 3 and 4.
Checks that the 15KHz and 64KHz clocks have the proper phase after exiting initialization mode.
Note: There is a one-cycle leeway in this test to account for temperature sensitivity in some systems.
Checks basic operation of PIA registers. This test will fail if port output and data direction registers are not distinct.
Checks for correct functionality of PIA port control bits 6 and 7, which cannot be set under program control.
Checks for the correct value being returned from undocumented GTIA registers.
Checks that the GTIA can read data bytes fetched by the 6502 CPU when the P/M DMA bits in ANTIC DMACTL and GTIA GRACTL do not match.
Checks for correct function of the GTIA CONSOL bits, particularly whether GTIA can pull the console switch lines low as well as read them. This test will fail if you press console buttons during the test.
Checks for basic functionality of player/missile collision registers.
Checks for correct collision behavior between players/missiles and the playfield when GTIA modes are active.
Checks that the same player or missile can be displayed multiple times in the same scanline under CPU control.
Checks for the correct scanout pattern when the size of a player is changed in the middle of display of that player.
Checks for correct behavior and timing for enabling and disabling GTIA modes in the middle of a scanline using the PRIOR register.
Checks for correct GTIA function in the $D020-D07F region. The $D080-D0FF region is not checked to avoid tripping some GTIA-based expansions.
Checks for correct behavior of the XL MMU, including swapping in and out the kernel, self-test, and BASIC ROMs, as well as writing to ROM. This test will fail if run on a 400/800 or if a memory expansion is installed that disables the BASIC or self-test ROMs; these are expected failures.
The "enable illegal instructions" option should be enabled and the "stop on BRK instruction" diagnostic option should be disabled under System > CPU Options... before running the test suite.
The MMU: XL Banking test will fail if the memory setting is set to 576K or 1088K, as those modes omit the ability to swap in the internal BASIC ROM. This is an expected failure.
The UseBasic setting must be enabled in order for the MMU: XL Banking test to pass. Otherwise, Atari++ will not allow Acid800 to bank in the BASIC ROM through PIA port B. This is an expected failure.
This version of kat5200 does not support the full set of undocumented 6502 opcodes and will report a crash during the undocumented opcode test on default settings. Use the options menu in Acid800 to disable the undocumented opcode test.
The Acid800 test suite is licensed under the MIT license.
Altirra Acid800 test suite
Copyright (C) 2010 Avery Lee, All Rights Reserved.
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.