Ethernet transmission

In the last post I described how receiving ethernet frames was working.

Transmitting ethernet frames looked like it should follow a fairly similar process, so I spent a little time working on this.

I have structured it so that full-duplex operation is possible, with separate state machines controlling transmission and reception.

The first step was to see if I could cause anything to come out of the ethernet port at all.  Here I hit a bit of a problem, because all modern ethernet cards automatically check the ethernet Frame Check Sequence (FCS), and discard frames that are bad.  This meant that I would need to implement the FCS, and form completely valid ethernet frames first-up.

I much prefer to be able to make incremental advances, knowing that I have addressed particular steps as I go along, so I really wanted something that would let me receive invalid ethernet frames.  Then it dawned on me that the solution was to connect two C65GS's together, since the ethernet receive side doesn't (yet) check the FCS or pretty much anything else about the frames when receiving them.

This also means that I can check things without having to worry about one end feeding crazy bonjour packets all the time.

This let me quickly confirm that in fact nothing was coming out of the ethernet port with my first attempt.

I took a guess that the transmit side might not start transmitting if you don't immediately start with the ethernet preamble code.  After fixing that, and with back-to-back Nexys4 boards running as C65GSs, I was able to cause simple frames to be sent from one to the other.  I just stuffed a sequence of bytes in to the transmit buffer at $FFDE800, and then set the frame length in $FFDE043 - $FFDE044, and then wrote $01 to $FFDE045, and voila, the frame was sent to the other side with a reassuring blink on the ethernet led:

.sffde043 ff 00 01

You can see the momentus frame as received at the other end here.  Sadly no "Watson, come here" or "One small step" here.

.Mffde800                                                       
 :FFDE800 FE 00 5C C2 76 86 0A 0B 0D 0D 0E 0F 07 08 09 01
 :FFDE810 02 03 04 05 07 07 08 09 02 02 03 04 05 06 00 00
 :FFDE820 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE830 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE840 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE850 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE860 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE870 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE880 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE890 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8A0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8B0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8C0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8D0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8E0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE8F0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 :FFDE900 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

The first two bytes are the received frame length (I can see that there is an out-by-one error here somewhere, my guess is on the transmit side), then the next four are the FCS as I calculate it as the packet is received, then the packet data starts after, from 0A onwards.  I think I am calculating the FCS incorrectly, as it never matches that of any real received frame sent by an actual computer, so I will need to look into that.

But anyway, this is good progress.

Starting work on the ethernet adapter

As I mentioned in an earlier post we have a student working on the ethernet controller for the C65GS using the on-board 10/100mbit ethernet adapter on the Nexys4 board.

We spent a bit of time yesterday understanding how it works, and it is very pleasing that less than 24 hours later I was able to receive this ethernet frame with the C65GS connected by ethernet to my Mac:

 :FFDE800 FF FF FF FF FF FF C8 2A 14 08 DA E2 08 00 45 00 ................
 :FFDE810 01 4E 9C 70 00 00 FF 11 FC DC A9 FE CD 54 A9 FE ................
 :FFDE820 FF FF EB 5D 13 8A 01 3A 33 D6 44 52 49 4E 45 54 ..........DRINET
 :FFDE830 54 4D A9 FE CD 54 C0 0E 00 00 00 3E 39 63 30 31 TM..............
 :FFDE840 61 38 63 30 2D 31 36 37 36 30 39 38 38 39 31 00 ................
 :FFDE850 00 00 00 00 00 00 00 00 00 00 00 00 69 71 6E 2E ............iqn.
 :FFDE860 31 39 39 35 2D 31 32 2E 63 6F 6D 2E 61 74 74 6F 1995.12.com.atto
 :FFDE870 74 65 63 68 3A 78 74 65 6E 64 73 61 6E 3A 73 65 tech:xtendsan:se
 :FFDE880 72 2E 63 30 32 66 38 72 6D 78 64 68 32 68 0A 00 r.c02f8rmxdh2h..
 :FFDE890 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 

Clearly this is not a very interesting ethernet frame for our purposes, but what is clear is that it is locking onto the frame preamble and receiving the bits and putting them together all correctly.

There is no ethernet checksum being performed yet, and it isn't possible to send frames, either.  These are things I will likely work on when I get a chance.

In the meantime the student, now joined by another student, who will work on writing some test software for the ethernet controller.

Once it is all working, then we will look at adding RR.net emulation registers so that existing software can use the ethernet interface.

Why hardware is hard

I have finally found and fixed the problem that was stopping most (but not all) CIA IRQs in the redesigned CPU.  The result was the cursor would blink very slowly.

I had bashed away for several days on the CIA, to try to figure out what was going wrong, with no luck.  I wasn't able to reproduce the problem in simulation, so I was pulling my hair out.

So I progressively added more and more instrumentation that revealed that the CIA was resetting the interrupt status register, and that it seemed to be doing so at the request of the CPU.  This should only happen if the CPU is reading from $DC0D.

It was about then that the realisation dawned on me that because the memory controller on my CPU has separate channels for RAM, IO and other types of memory, it is possible for the IO bus to still be presenting the instruction to read from the last accessed IO address.  Indeed $DC0D is the last IO address touched in the C64's IRQ routine, and thus the problem.

This also explains why the problem would go away if I accessed any IO-mapped memory, even from the serial monitor.  In retrospect, I should probably have reflected on this a little more deeper, especially the fact that the CIA design hadn't changed, but my CPU design HAD changed since it was last all working.

Anyway, this was a reminder of why hardware design is hard: it does exactly what you ask it, and keeps on doing it until you ask it to stop.  Getting that right all of the time takes a lot of careful attention and testing.

First speed test of 48MHz CPU

I am continuing to fight with getting the reimplemented CPU and VIC-IV all settled down, however things are getting much closer.

As the following image implies, it can run the C64 ROM (not yet C65 ROM -- it gets stuck in the DOS somewhere).  It should be noted that the CPU performance here is not final, and some instructions might end up faster or slower than depicted here.  That said, the CPU is certainly quite a bit faster than the old 32MHz one.

44.36x is almost exactly 48mhz/32mhz = 150% the speed of the old CPU at 28.93x.  Pleasingly this is before I do anything to optimise the performance.  Also, whereas the old CPU filled the FPGA to capacity, with the new CPU about two-thirds of the FPGA remains free -- space for implementing sprites, a 1541 and other goodies as I get the chance.

Speaking of optimisations, one that I may attack in the not too distant future is a stack cache that allows RTS to execute in just 1 cycle.  While it will make some impact on the SynthMark64 score, it is more interesting for real-life work loads where JSR/RTS are very common instructions.  But otherwise there is no caching anywhere in this -- it is all raw, predictable cycle times, which helps make it feel like a simple 8-bit computer, albeit a very fast one.

What isn't entirely obvious here is that keyboard input has broken for some reason, with my PS/2 keyboard reader failing to detect key-release events.  Also, for some reason the CIA interrupts are not always happening as often as they should.  Combined with not being able to use the C65 ROM, this means I had to side-load SynthMark64 via the serial monitor, start it directly from the serial monitor, and then use the serial-monitor to stuff the ENTER key-press into the keyboard buffer.  So there is still a bit to go, but at least it feels like I am getting somewhere.

Debugging the new CPU

Debugging of the re-implemented CPU continues, and is hopefully close to complete.  I thought I would describe some of the process I have followed.

The real secret to debugging anything is discoverability, that is having the means to work out what is happening so that you can tell not only whether the result is correct, but how the result is being calculated.

With a hardware design this can be rather annoying, because the time it takes to make a trivial change, resnythesise and test the design can be of the order of an hour.  Assuming that you correctly expose the thing you are trying to debug, that means you can examine (and hopefully fix) at most a dozen or so defects per full day of effort. Not Good.

Fortunately there are simulation tools for VHDL that let you debug without having to go through the whole synthesis process, and thus reduce the time to examine a defect from hours to minutes.  While this has limitations, for example, to debug the SD card interface in simulation I would need to write an SD card simulator, it is extremely useful, and I have made extensive use of the free and open-source simulation tool, ghdl.

The processor redesign basically consisted of gutting out the first implementation of the CPU and leaving just the shell that accesses the memory and interfaces with the serial monitor, which I have described in previous posts.  The serial monitor is extremely useful, because it allows reading and writing of all memory, as well as examining the processor state, and single-stepping the processor.

The first part was to re-do the serial monitor interface, because this needed an overhaul for the new processor architecture.  This was rather tricky, because simulating a serial connection feeding various commands in would take a fair bit of work, and the time scales of serial input means that simulation would be rather slow anyway.  So as a result I used some of the LEDs on FPGA board to provide some useful debugging output, and worked as carefully as I could to make sure that the code was likely to work.

The second and related step was getting the memory access stuff working again, and accessible via the reworked serial monitor interface.

These two steps took much longer than I had hoped, and were really frustrating.  In retrospect, it might well have been easier to make a simulator for serial input and used ghdl simulation to shorten the process a bit.

After this, I set about implementing a few simple instructions so that I could get single-stepping of the CPU through the serial monitor working.  This also turned out to take way longer than I would have liked, partly because the new CPU architecture uses 6502-style end-of-instruction pipelining which really complicated single-stepping.  I did get it working in the end.

Then it was on to implementing LDA, STA, JMP and a few other instructions to allow the writing of simple little test programs to confirm that the CPU was generally working.  At this point ghdl was useful to allow quick testing of the instructions and their interactions.

In the process of doing this, I realised that the debug output I was producing in ghdl was not as good as it could be.  Basically I was looking at hexadecimal instruction bytes and trying to decide if it was right or not.

It would be much easier to debug if I could get ghdl to show full instruction disassemblies as well, so in stead of just seeing 8D 0D DC, it would also show STA $DC0D.  Also, it would help enormously to know what memory access was happening each cycle, so that I could get an idea of exactly where an instruction was going astray.

I finally had time to implement this during the week, and now I can easily get output like:

MEMORY reading $FFFF654 = $A9
MEMORY reading $FFFF655 = $00
MEMORY reading $FFFF656 = $85
$F654 A9 00     lda  #$00          A:00 X:22 Y:33 Z:00 SP:01FF P:26 $01=3F  ..E-.IZ.  
MEMORY reading $FFFF657 = $20
MEMORY reading $FFFF658 = $A9
MEMORY reading $FFFF658 = $A9
MEMORY writing $0000020 <= $00
$F656 85 20     sta  $20           A:00 X:22 Y:33 Z:00 SP:01FF P:26 $01=3F  ..E-.IZ.  
MEMORY reading $FFFF659 = $91
MEMORY reading $FFFF65A = $91
MEMORY reading $FFFF65A = $85
$F658 A9 91     lda  #$91          A:91 X:22 Y:33 Z:00 SP:01FF P:A4 $01=3F  N.E-.I..

Actually the output has a little more information in it, but the above gives you an idea.

We can see a few things from this output.

First, the instructions seem to work, as we see the right values end up in the accumulator, and the correct value being written to the write address.

Second, we can see that there is a dummy read in STA, which is part of the design that allows 48MHz operation.  So for some instructions at least, we don't expect 48x performance.  Some of these might get improved down the track, but some penalty cycles will have to remain.

Thirds, we can see the 6502-style pre-fetching of the next instruction while the previous instruction is finishing off.

Armed with the ability to produce this kind of trace, I used the TTL6502 test program for 6502 processors, and by examining the simulation output was able to quickly find and fix quite a number of bugs.

The TTL6502 program only tests the original 6502 instructions, not any of the 4502 extensions.  So I have followed a bit of an ad-hoc process of writing little programs that use each of the new instructions, and verifying from the memory trace, register and flag values that all is well.  This has also turned up a great many bugs.

This is more or less where I am at now, fixing bugs with PHW (push word, immediate or absolute) and a few other remaining instructions.  Once that is done, we should hopefully be back to being able to boot the C65 ROM into C64 mode, and then soon after running SynthMark64 to get an idea of the speed of the new CPU.

Improved hardware scaler

Previously the pixel scaler for the VIC-IV allowed logical pixels to be any integer number of physical pixels in both X and Y directions.  This struck me as a little inadequate.  For example, for 80 column mode it meant either no side borders or enormous side borders.

So I have replaced these simple integer counters with fixed point counters that are used as divisors for the width of pixels. A value of 1 means that logical pixels will be 128 pixels wide, while a value of 255 means that logical pixels will be 1/2 a physical pixel wide.  This allows zoom factors from 0.5x to 128x, with very fine granularity at the smaller end of the range.  The following frame shows the smaller end of the range and just how fine the graduation is.  You really need to click on the image and zoom it in to see what is going on.  Memory is not uniformly initialised, hence the different textures that can be seen.

There is no sub-pixel sampling, so there will be aliasing effects.  Nonetheless, the result is much more flexible than it was previously.  When I get a moment I will adjust the 80-column display modes to use this with 2.5 physical pixels per logical pixel, so that the borders don't move when switching to 80 column mode -- unless of course the result looks too silly with the mix of fat and skinny pixels.

More work on new CPU, and some very skinny raster stripes

It still isn't very exciting to look at right now, but the CPU is getting closer to working properly.

I have found and fixed abut in the TRB (Test Reset Bit) instruction. This is a handy little instruction for clearing bits in byte.  For example, LDA #$01 / TRB $D030 will clear bit 0 in $D030, which on a C65 will bank out the second kilo-byte of colour RAM from $DC00 - $DFFF so that you can see the CIAs again.  The correct calculation for the result is (memory and (not A)), but I had (memory and A), which has the effect of reseting all of the bits except the one(s) you wanted reset.  Needless to say that wasn't working too well.

I also fixed some bugs with IO mapping.  In particular, the SD card controller is visible to the CPU again, and Kickstart even gets as far as loading the master boot record from the SD card.  There does seem to be an out-by-one error with the buffer addresses, such that the whole sector is rotated by one.

Here is Kickstart finding the SD card at 48MHz:

That looked a bit boring, so I wrote a little loop to do some raster effects:

This is the little loop:

loop     LDA $D052   ; VIC-IV physical raster line low bits (range 0 -  1199)
         CMP $D052
         BEQ *-3
         INC $D020
         DEC $D020
         JMP loop

  The loop should increment and decrement $D020 just once at the start of each raster line. However it looks like the compare instructions are using a fixed value, instead of the operand, which is why there are a few rasters on which there are no bars, while the rest fail to properly compare the raster number with itself.

This is due to a bug in the compare instructions, which I have yet to get to the bottom of. My gut feeling is that it is some sort of timing bug, where the wrong value is read from the bus.  I have seen it in simulation once or twice, which suggests that I should be able to analyse it fairly easily to find and fix the cause.

Meanwhile, it is interesting to look at the pattern and how narrow the stripes are.  They are actually almost half the width that they seem at first when you look closer, because of the adjacency of the bars on successive raster. The following image makes this a bit clearer:

The VIC-IV runs at 4x the CPU clock, so every four physical pixels corresponds to once CPU clock tick.  The logical pixels of the character generator are five logical pixels wide here, so one and a quarter CPU clocks wide.

INC and DEC take seven cycles on my CPU at the moment, due to the need to include wait-states to avoid back-to-back memory accesses at 48MHz.  This should equate to 7x4 = 28 pixels, or about five and a half logical pixels, just over half a character wide, which is pretty much what we are seeing.

On a real C64 the same bars would be almost 10x wider, at six characters or 48 pixels wide.  So even allowing for the massively higher pixel clock on the C65GS (192MHz versus 8MHz on the C64), there is certainly scope to do some pretty interesting tricks.  Vertical raster bars and split screens should both be quite possible, although there are probably easier ways to get the same effects.