Sunday 9 September 2018

Freezing and the MEGA65 System Partition -- part 1

My current plan for the MEGA65 is that the core operating system will be focused around a hardware integrated freeze function, a bit like having a cartridge like the Action Replay built in.

Where we are going, you don't need one of these, as much as I love my Datel Action Replay (TM) when using a C64.


The idea here is that if you want to change disk images, or load a program, or switch to another program, these can all be logically handled from in the freezer.  Things like the current disk image are just flags in the Hypervisor's process descriptor for the currently running program.  Suspending and switching to other programs is of course the bread and butter of what a freeze cartridge does.

I started work on the freeze code a while back, and have just this week had a bit of time to come back to it, and got it to the point where freezing (mostly) works, although the un-freezer is yet to be written.

One of the challenges of the MEGA65 is that we have purposely made the core operating system only 16KB, partly to save hardware resources, and partly because the smaller something is, the less bugs it can hide.  Yet at the same time, we want to have a very functional and fun to use.

The way we are solving this dilemma of needing to be frugal versus making something that is a real joy to interact with, is that we are separating the Hypervisor functions from the freeze utility.  The Hypervisor will have functions to freeze and unfreeze, but will not have any user-interface built in.  Instead, when you hit the freeze/menu button, it will freeze the system to SD card, and then load the freeze menu program from SD card, in the form of a frozen system image, and give it special permission to make direct access to the SD card, so that it can look through the list of frozen programs, including pulling up the labels and thumbnail images that we can associate with them, and let the user choose one to load.  It will also let you change the floppy disk image attached (and switch to/from using the real floppy drive).


To make this work with the tiny hypervisor, we need to have an area of the SD card set aside for freeze images, and also for helper programs like the freeze menu.  We could have this in the FAT32 file system, but then we have to worry about whether files are contiguous, and update the FAT entries, all things that are rather a pain to do in a 16KB ROM.  Thus, instead, I have created the concept of a system partition for the MEGA65, that contains the freeze and service program slots, as well as a general configuration area, e.g., to remember whether you want 50Hz or 60Hz video on boot, and whether you want to enable 1351 emulation for Amiga mouses, and all the other sorts of things you want to remember between resets.

The system partition is created as a normal master boot record (MBR) partition, but with partition type 65 ($41).  Apart from this being the most obvious partition type number for us to want, it has the advantage that it isn't really used for anything on anything resembling a modern system.  Wikipedia says it was used for "Old Linux/Minix (disk shared with DR DOS 6.0) (corresponds with $81)", or "PPC PReP (Power PC Reference Platform) Boot".  Neither of these are likely to cause us any grief in typical usage.  Indeed, Linux, Windows and OSX all ignore partition type $41. This means we can have a MEGA65 system partition and a FAT32 partition for data storage, and if you take the microSD card out of the MEGA65 and put it into any modern computer, you will see the FAT32 partition, making it easy to transfer files, but won't be bothered by seeing the system partition.  If you want to modify the system partition, then this will be done using the existing MEGA65 system setup utility from the Hypervisor's Utility Menu, or by using the freeze menu.  So that's all fairly nicely sorted.

I hinted at the ability to have more than one service program installed in the system partition.  This is not accidental. The idea is that there can be many such programs installed on a MEGA65 system, and which can then be called from any program running on the MEGA65 in any mode.  This will happen by setting up a Service Program Call, which consists of setting up a little area of memory with the name of the service you want to call.  The Hypervisor then looks for a frozen service program with the same name, and loads it in, keeping only the first part of memory from the calling program as a transfer area. It can then do its job, say, downloading a file from a URL to SD card, and then updates the transfer area to say whether it succeeded or failed, and then calls a return-from-service-program Hypervisor trap that reverses the process.  This is a little, but only a little, like the Intent system on Android for inter-process communications.  Key differences include that the MEGA65 is never actually running multiple programs at the same time.

Writing the freeze routine was a bit interesting.  As mentioned, it has to be compact.  There is also the extra challenge that the MEGA65 has a lot more IO and little bits of state hanging around the place that need to be saved, including the SD card registers.  The usual path a freeze cartridge takes is to have some extra memory, so that you can stash a few things there while saving state. We have done the same, using the otherwise unused first 3KB of the 4KB BRAM we use for the SD card sector buffer.

To keep things compact, I made a list of memory regions and their sizes that have to be saved to the SD card, so that the table of memory regions to save/restore when freezing/unfreezing takes seven bytes each: four bytes for the
first address, followed by three bytes of length, allowing regions of up to 16MB to be represented.  I then added an eighth byte that is used as an entry in a look-up table that can contain special routines to prepare before saving the area.  This is used, for example, to save each of the four VIC-IV palette memories, only one of which can be mapped at a time.  The result is that the entire description of what to save, and how it should be done is a list of these regions.  Here is that list at the time of writing (we know there are a few things missing for now, and will add them in. Bonus points for working out what they are).

    ; SDcard sector buffer (direct access) and registers
    .dword $ffd6000
    .word $0290
    .byte 0
    .byte freeze_prep_stash_sd_buffer_and_regs

    ; SDcard sector buffer (F011)
    .dword $ffd6c00
    .word $0200
    .byte 0
    .byte freeze_prep_none

    ; Process scratch space
    .dword currenttask_block
    .word $0100
    .byte 0
    .byte freeze_prep_none
   
    ; $D640-$D67E hypervisor state registers
    .dword $ffd3640
    .word $003F
    .byte 0
    .byte freeze_prep_none

    ; VIC-IV, F011 $D000-$D0FF
    .dword $ffd3000
    .word $0100
    .byte 0
    .byte freeze_prep_none

    ; 128KB RAM + 128KB "ROM"
    .dword $0000000
    .word $0000     
    .byte 4          ; =4x64K blocks = 128K + 128K
    .byte freeze_prep_none   

    ; VIC-IV palette block 0
    .dword $ffd3100
    .word $0300
    .byte 0
    .byte freeze_prep_palette0

    ; VIC-IV palette block 1
    .dword $ffd3100
    .word $0300
    .byte 0
    .byte freeze_prep_palette1

    ; VIC-IV palette block 2
    .dword $ffd3100
    .word $0300
    .byte 0
    .byte freeze_prep_palette2

    ; VIC-IV palette block 3
    .dword $ffd3100
    .word $0300
    .byte 0
    .byte freeze_prep_palette3
   
    ; $D700-$D7FF CPU registers
    .dword $ffd3700
    .word $0100
    .byte 0
    .byte freeze_prep_none
   
    ; 32KB colour RAM
    .dword $ff80000
    .word $8000
    .byte $00
    .byte freeze_prep_none

    ; End of list
    .dword $FFFFFFFF
    .word $FFFF
    .byte $FF
    .byte $FF


The labels in bold are the index entries for the various setup routines. These are actually called by using a 65C02 jump table, using the following:

     jmp (freeze_prep_jump_table,X)

freeze_prep_jump_table:
    .alias freeze_prep_none 0
    .word do_freeze_prep_sdcard_regs_to_scratch
    .alias freeze_prep_palette0 2
    .alias freeze_prep_palette1 4
    .alias freeze_prep_palette2 6
    .alias freeze_prep_palette3 8
    .alias freeze_prep_stash_sd_buffer_and_regs 10
    .word do_freeze_prep_palette_select
    .word do_freeze_prep_palette_select
    .word do_freeze_prep_palette_select
    .word do_freeze_prep_palette_select
    .word do_freeze_prep_sdcard_regs_to_scratch


Where the X register has been set to the value of the eighth byte.  This is why the indexes are all even values, so that the jump table works without extra fiddling.   The .alias directives setup the value required to call the routine, which we do there in the jump table, so that it is easier to keep them in sync, if we modify things.

What is required in each setup routine differs somewhat. The palette ones are interesting, because all they need to do is to modify the VIC-IV memory map.  To save some space, the same routine is used for each, and uses the value of the X register that was used to do the jump into the jump-table:

do_freeze_prep_palette_select:
    ; X = 6, 8, 10 or 12
    ; Use this to pick which of the four palette banks
    ; is visible at $D100-$D3FF
    txa
    clc
    sbc #freeze_prep_palette0

    ; A now contains 0 if palette0, 2 if palette1 etc
    ; Shift it left six bits, so that that value 0-3 goes into
    ; the bits that control which palette bank is currently
    ; memory mapped
    asl
    asl
    asl
    asl
    asl

    ; Update VIC-IV memory map
    sta $d070
    rts

Thus we need only 13 bytes to implement four of these helper routines.  Some are more complex, in particular, the one that saves the state of the SD controller, since for that we have to save some registers and the SD card sector buffer itself into the 3KB RAM dedicated to the freezer I mentioned earlier.  That said, it isn't too complex, it is really just a case of copying the memory from the SD controller registers and sector buffers.

So that's the overall structure for how we are implementing freezing, and the result is reasonably compact and maintainable, which are our primary goals.  That isn't to say that we didn't have (and don't still have) bugs in the code.  To help track those down, and to make sure that programs are being frozen properly, we began writing a little Linux utility that reads an SD card and looks for a MEGA65 system partition and is able to find the freeze slots,  and save them out to files.  This helped us find some problems, such as if a single memory region was >64KB, the same 64KB was being saved repeatedly. That utility is still very much a work in progress, which we expect will get refined over coming weeks.


In terms of freezing speed, at the moment, it takes about one second to freeze using a class-10 microSD card.  This is mostly because our SD controller doesn't (yet) support sequential block writes.  This means the SD card writes an entire ~64KB flash block every time we write a 512 byte sector.  As a result, it spends perhaps 100x more time writing to flash than is required, and will wear the flash out that much faster too, in all likelihood. For both these reasons, we intend to implement sequential writes to the SD card.  Given our SD card interface is capable of about 3MB/sec, and freezing the entire state of the MEGA65, assuming all memory etc is being used, requires somewhere around 0.5MB, this should let us get the freeze time down to about 1/6 second, which seems like it should be pretty good.


No comments:

Post a Comment