On the C65, holding down the commodore key during reset or power on boots to C64 mode instead of C65 mode, similar to the C128.
Alternatively, if you hold down the run-stop key instead, it drops you into a machine language monitor, without clearing memory. This is quite a nice feature.
However, when I had tried this with previous builds, the display of hexadecimal characters was all wrong. There would be one hex digit, followed by a strange character. A screen shot is below:
As you can see things are quite wrong. The colour changes are because printing certain characters on Commodore 8 bit computers causes the drawing colour to change.
So I set about discovering where this problem was.
I knew the monitor sits in its own 8KB part of the ROM as a discrete program. I also knew that it prints a ">" just before printing a hexadecimal value. From experience, I also knew that the hex printing routine would likely have four LSR A instructions to shift the high nybl down. This, I figured would help me to find the exact place in the code where things were going wrong. I hoped to then use the serial monitor debug interface to step through the code and see where it goes awol.
It didn't take too much searching to find the routine in question. Just looking at it, I was immediately sure that it was a hex printing routine. Apart from the four LSR's, it had the tell-tale comparison with $0A to work out whether it is printing a digit or letter. Here is the routine:
69EB 48 PHA
69EC 63 07 00 BSR $69F5
69EF AA TAX
69F0 68 PLA
69F1 4A LSR A
69F2 4A LSR A
69F3 4A LSR A
69F4 4A LSR A
69F5 29 0F AND #$0F
69F7 C9 0A CMP #$0A
69F9 90 02 BCC $69FD
69FB 69 06 ADC #$06
69FD 69 30 ADC #$30
69FF 60 RTS
Staring at it for a few minutes I realised that the problem was the BSR instruction was not doing anything.
BSR is a cross between JSR and the branch instructions on the 6502. Like JSR, it pushes the return address to the stack, so you can use RTS to resume execution, but like the branch instructions, it uses relative addressing.
This struck me as odd, because I knew I had implemented that instruction. However, I hadn't tested it, because I am still in the process of writing a comprehensive CPU test suite for the 4510. After staring at the relevant VHDL code, I realised the logic error that was preventing the BSR instruction from ever being executed.
Basically BSR uses 16-bit relative addressing, and before the check for the BSR instruction there is a case for all relative addressing instructions to decide whether to take the branch or not. Basically BSR was being treated as a conditional branch with an impossible condition. A quick bit of rearrangement, so that BSR is handled first, and suddenly hex output was working, and the monitor now seems fully functional:
I had hoped that this would also fix the problem with C65-mode BASIC ignoring all commands, but no such luck. There must still be at least one more CPU bug in there. All the more reason to get that test suite written.