Friday 22 December 2017

Automatic VIC-II Compatibility Regression Testing - Part 1

One of the challenges of implementing the MEGA65 is to be sure that we have compatibility with the VIC-II and other custom chips in the C64.  This gets more complex as we fix bugs and add missing functions, as we might accidentally introduce a regression. 

This is of course why test cases are very useful, and there are a bunch of test cases as part of the excellent VICE emulator suite. However, from the ones I have looked at, they are rather specialised, and there is no trivial way to run them all, and get informed when one or more of them fails.  (It is possible that I have the wrong end of the stick here, and that this can be done with VICE, and I would welcome correction on this point.)

I have decided to fix this state of affairs, and write a single program that runs through a large number of function and compatibility tests, and provides meaningful failure messages, so that whenever we build a bitstream, we can check to make sure that nothing has gone backwards.

My expectation is that this program will grow in fits and starts according to our needs, and may end up being a set of automatically chaining test programs, similar to the 6502 test suite for the C64, that has programs that test a specific instruction, and then load the test for the next instruction when done.

Anyway, I have begun implementing the first tests to help track down a problem with sprite positioning in the new 800x600 video modes.  It starts by checking sprite to sprite collision and then sprite-to-character collision, in such a way that it can work out where sprites are being drawn,relative to the text display.

The program purposely is C64 compatible, so that I can make sure that the tests run the same on a real machine (or in VICE for the simpler stuff, until I get an SD-IEC or Ultimate 1541).  Here is what it looks like running in VICE so far:
The test that fails might actually be wrong -- I am currently researching to find out of a real C64 detects sprite collisions that happen behind the borders. My gut feeling is that it probably does, but I am not yet sure.

Anyway, it is a starting point, and I will post an update when it is somewhat more matured.

6 comments:

  1. I think two cores may be better than one in this case. You can make one core to implement a C64 100%, and the other core to implement a C65/C65+. This core does not need to be 100% C65 compatible because of the lack of existing C65 software. I think it should have a mode that is as close to a C65 as is practical, and also have a mode that unlocks the full performance. We know the C65 prototype was not very compatible with C64 software due to its hardware changes. Why try to make your core more compatible? I think you are adding unnecessary complexity. I also think the end result will be less faithful to the C65. It seems like it is very challenging get to 100% C64 and 100% C65 in one core. If you have two cores then you will have more resources available for making the C65 even more powerful because you will not need to include all of the C64 stuff in the main core. You will also have more resources in the C64 core to add things like a true 1541 implementation. Each core will be less of an in-between compromise. The C64 core could be loaded from the SD card and reconfigure the FPGA in a few seconds. Anyway, I hope I don’t sound negative because I think you are doing awesome work and can’t wait to see (and get my hands on) the final computer.

    ReplyDelete
    Replies
    1. Hi Nathan,
      The compatibility improvements we are making are only those that are practical to do so. For example, matching 6502 cycle timing, making sure that idioms for $D019 work and so on. This is part of our objective of making the MEGA65 be what the C65 should have been. In this regard we are aiming for "very good" C64 compatibility, but not perfect. We want people to be able to use the C65 core for most C64 work, since that is the idea of the machine, at least in our minds. As for 1541s etc, we hope to have enough spare space in the FPGA to run a 1541, but you will also be able to connect one externally, as per normal, or indeed, an sdIEC or possible an Ultimate 1541 II or II+ or similar, since the cartridge port will also be "mostly compatible" instead of "the wrong number of pins". There are trade-offs in it all, and in some sense, part of the fun is working through those. Of course, because it is all open-source, people are welcome to disagree with our choices, and implement their own :)

      Delete
  2. If you have a quick look at the 1541 Ultimate source code, does it look like you can reuse (parts of) it to implement a 1541 inside the Mega65? I would make 1541 emulation way more practical, since Gideoń's implementation is close to perfection, I really don't know of any software that runs on a real 1541, but not on a 1541U2.

    ReplyDelete
    Replies
    1. If it is open-source, we would be very happy to make use of it (and of course, because the MEGA65 is open-source, the offer is a two-way one).

      Paul.

      Delete
  3. It is open source. The Subversion repository of the 1541 Ultimate is available here:

    https://xp-dev.com/svn/1541UltimateII/trunk/fpga/

    ReplyDelete
    Replies
    1. Ah, cool. Thanks. We will definitively have a look at that when we get to adding 1541 support internally in the MEGA65.

      Paul.

      Delete