In my previous post, I implemented a facility for accessing the brand new Shared Resources area of the MEGA65's System Partition.
This facility has been created to provide rapid access to content anywhere in large files --- initially unicode pre-rendered fonts for the MEGAphone. Using a traditional file-system would require tracking file-system state, and traversing file-system structures. This would add latency for time-critical operations, like retrieving individual character glyphs when rendering SMS messages.
This area works by implementing a simple extent-based file list in the system partition. The shres and shresls tools exist for linux in the mega65-tools repository to populate and query the contents of this area on a MEGA65-formatted SD card.
Only the latest development version of MEGA65 FDISK will format an SD card to include a shared resource area.
test_897.c in mega65-tools has unit tests for this functionality, including reporting if your HYPPO version is too old, or there doesn't seem to be a shared resources area defined in your system partition.
So now we need to tie all this together is some code that could eventually go in mega65-libc that provides a simple API for accessing it. I'm thinking something like:
char shopen(char *resource_name,unsigned long long required_flags, struct shared_resource *file_handle) -- open
shread(unsigned char *ptr, unsigned int count, struct shared_resource *file_handle) -- Read a slab of bytes from the opened resource from the current offset.
shseek(struct shared_resource *,unsigned long long offset, unsigned char whence) -- Seek to arbitrary point in the file.
unsigned int shdopen() -- open directory handle to shared resource area
sddread(unsigned long long required_flags, unsigned int *directory_handle, struct shared_resource *dirent) -- read next directory entry that matches. The resource is implicitly an opened resource (since "opening" is absurdly light weight, the way I'm doing it).
The shared_resource struct will simply contain the filename, flags, starting sector in the shared resource area, its length, and current seek position (initially 0):
#define MAX_RES_NAME_LEN 256 #define SEEK_SET 1 #define SEEK_CUR 2 #define SEEK_END 3 struct shared_resource { char name[MAX_RES_NAME_LEN]; unsigned long long flags; unsigned long long length; unsigned long long first_sector; unsigned long long position; };
To test this API, I'm just going to start writing a wrapper program in the megaphone-modular repository in src/telephony.
Okay, so in the process of setting that up, I'm discovering that CC65 mainline doesn't have support for the 45GS02 instruction set, so I can't use LDQ and STQ directly. I guess I'll just implement those as the underlying instruction sequences.
Next challenge is that the SD card keeps getting stuck busy. One of the subtleties involved in this is that the SD card interface _does_ lockup if you ask it to read a sector while it's already reading a sector. We should probably fix that, but for now I have to live with it.
To deal with this, I've made the shared resource API functions check the SD card status, and fail with an error if it's busy. So it should never be calling a SHRES Hypervisor trap when the SD card is busy (remember that the SHRES Hypervisor trap does the sector read request, so this is vital).
char do_shres_trap(unsigned long arg) { // Fail if SD card is busy if (PEEK(0xD680)&0x03) { printf("SD card is busy\n"); return 1; }
// Check trap response in P return (shres_regs[4]&0x01) ^0x01;
}
Basically the wrapper is the only way I call the trap, and it should reliably fail if the SD card interface is already busy.
So what's going wrong here?
Okay, so it looks like it's trying to read an invalid sector from the SD card for some reason now. The call to shdopen() always asks for sector 0 of the shared resource area:
shared_resource_dir shdopen() { char i; if (do_shres_trap(0)) return 0xffff;
So provided that shres_regs is getting set properly in the wrapper it calls, it should be working.
I _think_ there might be something funny where the HYPPO trap is not adding the SHRES start offset to the $D681-4 SD card registers properly.
readsharedresourcetrap: cpq syspart_resources_area_size bcs bad_syspart_resource_sector_request ldq syspart_resources_area_start adcq $d681 stq $d681 ;; Ask SD card to read the sector. lda #$02 sta $d680 ;; Note that we don't wait for the request to finish -- that's ;; up to the end-user to do, so that we don't waste space here ;; in the hypervisor, where space is at an absolute premium. jmp return_from_trap_with_success
Ah! There is indeed a logical error here. It should look like this:
readsharedresourcetrap: cpq syspart_resources_area_size bcs bad_syspart_resource_sector_request adcq syspart_resources_area_start stq $d681 ;; Ask SD card to read the sector. lda #$02 sta $d680 ;; Note that we don't wait for the request to finish -- that's ;; up to the end-user to do, so that we don't waste space here ;; in the hypervisor, where space is at an absolute premium. jmp return_from_trap_with_success
Yay! With that fix, I can now iterate over the (slightly random) set of files I put in the shared resource area of my SD card:
So perhaps I'll next make it try to cat the README.md file to the screen, just as a proof-of-concept --- and make it use the API, forcing me to implement all of the API.
// Make sure SD card is idle if (PEEK(0xD680)&0x03) { POKE(0xD680,0x00); POKE(0xD680,0x01); while(PEEK(0xD680)&0x3) continue; usleep(500000L); }
if (shopen(filename_ascii,0,&dirent)) { printf("ERROR: Failed to open README.md\n"); return; }
{ unsigned int b = 0; printf("Found text file\n"); while(b = shread(buffer,128,&dirent)) { for(i=0;i<b;i++) { if (buffer[i]==0x0a) putchar(0x0d); else putchar(buffer[i]); }
} }
return; }
And it works! And I can even use shseek() to print just a bit of the text file.
{ unsigned int b = 0; printf("Found README.md\n"); shseek(&dirent,-200,SEEK_END); while(b = shread(buffer,128,&dirent)) { for(i=0;i<b;i++) { if (buffer[i]==0x0a) putchar(0x0d); else putchar(buffer[i]); }
} }
Running this we can get something like this:
In short, it works. So now it's time to make a pull-request into mega65-libc for this new API, and then we're done, and I can go back to working on making the actual unicode pre-rendered fonts for the MEGAphone telephony software!
In another blog post I'm writing at the moment, I'm exploring how I will do the UI for the new MEGAphone work. Part of that involves me figuring out how I can possibly support the full Unicode gammut -- including emojis, that are not more or less a required part of SMS communications.
In that post, I considered putting them into files on the FAT32 file system, or stashing them in a set of specially formatted D81 or D65 disk images. But all of those approaches require more than 1 SD card sector read to fetch a random glyph, and in some cases many such requests.
So I came up with the idea of putting linear slabs of shared resources into the system partition, and having a Hypervisor call that lets a user program ask for an arbitrary 512 byte SD card sector of that data. For fonts in the MEGAphone software, this would allow the program to work out and store the sector offset within the shared resources for the font(s) it needs to use, and then just adding an offset to that to retrieve the SD card sector with the required font glyph to load into chip RAM.
In this blog post I'm going to design and implement this facility in the hypervisor, and make a test program that will let me check that it's all working. From there, I'll work on some kind of utility that will let me modify the contents of the system partition to install the fonts. That might end up being a Linux-based utility that works directly on the SD card for speed --- since a full Unicode font will be about 300MB in the pre-rendered format that I'll be using (more about that in the other blog post, when it's done).
I'm going to track the addition of this functionality into HYPPO via Issue #897.
So let's start by making the system partition code in the Hypervisor read and store the starting sector and size of this slab.
Now I have to remember how I made it possible to update the hypervisor using m65 -k. From memory this can be done safely from the MEGA65's READY prompt, as that means that it's in user-space, and thus it's safe to overwrite the hypervisor.
... except that that doesn't seem to do it.
Okay, so because the Hypervisor RAM is only writeable when the CPU is in Hypervisor mode, we can't do it from the READY prompt any more. But we also need it to happen in some situation where the Hypervisor isn't really running any code, since otherwise the CPU will go off into la-la land, and bork things.
I am using an old version of m65, though. Maybe in newer versions -k looks after this, e.g., by setting the CPU to single-step mode, then causing the CPU to start a Hypervisor trap, and then stopping the CPU once in Hypervisor mode, writing the new version of the Hypervisor code, then resetting and resuming the CPU, so that it boots the new HYPPO fresh. If not, that would be the way to do it.
I'm tracking this via Issue #222, and pull request #223.
Now with m65 recompiled with that fix, I can quickly and easily test HYPPO builds. That's already made it _way_ more comfortable and convenient to find and fix a bug in my initial implementation of the of the shared resource thing.
So now I can implement the HYYPO trap that tries to read the sector. Because we are going to use A,X,Y and Z as the Q pseudo-register to make it easy for a user program to access, we will have to use up a whole trap address, but that's okay.
So far, the trap code looks like this:
readsharedresourcetrap: stq $d681 cpq syspart_resources_area_size bcs bad_syspart_resource_sector_request ldq syspart_resources_area_start adcq $d681 ;; Ask SD card to read the sector. lda #$02 sta $d680 ;; Note that we don't wait for the request to finish -- that's ;; up to the end-user to do, so that we don't waste space here ;; in the hypervisor, where space is at an absolute premium. clc rts bad_syspart_resource_sector_request: sec rts
In principle, that should be all we need.
The next step is to make a unit test for this, checking that it does cause a sector to get read, and that requests for out-of-range sectors fails.
I'll use the unit test framework that we have in mega65-tools.
Those tests use CC65, which I can't remember it's call convention for 32-bit args. So I'll just stash the 32-bit resource sector number into some convenient memory location that it doesn't use. Maybe $7F0-$7F3, i.e., the end of the screen RAM.
The test will then make the HYPPO call, and store the register values from HYPPO back where it read them from. Like this:
.export _test_resource_read
.p4510
.segment "CODE"
.proc _test_resource_read: near lda $7f0 ldx $7f1 ldy $7f2 ldz $7f3 sta $d644 ; Trigger Hypervisor trap nop ; CPU delay slot required after any hypervisor trap sta $7f4 stx $7f5 sty $7f6 stz $7f7 php pla sta $7f8 rts .endproc
Pulling it all together, we can now easily run tests like this:
paul@bob:~/Projects/mega65/mega65-tools$ make src/tests/test_897.prg && m65 -u -4 -r src/tests/test_897.prg /home/paul/Projects/mega65/mega65-tools/cc65/bin/cl65 --config src/tests/tests_cl65.cfg -I src/mega65-libc/cc65/include -O -o src/tests/test_897.prg --mapfile src/tests/test_897.map --listing src/tests/test_897.list src/tests/test_897_asm.s src/tests/test_897.c src/mega65-libc/cc65/libmega65.a 2025-06-07T07:10:52.445Z NOTE MEGA65 Cross-Development Tool 20250531.20-222fix-ee98359 2025-06-07T07:10:52.454Z NOTE #0: /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-07T07:10:52.454Z NOTE selecting device /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-07T07:10:52.660Z NOTE loading file 'src/tests/test_897.prg' 2025-06-07T07:10:52.841Z NOTE running 2025-06-07T07:10:52.841Z NOTE Entering unit test mode. Waiting for test results. 2025-06-07T07:10:52.841Z NOTE System model: UNKNOWN MODEL $06 2025-06-07T07:10:52.841Z NOTE System CORE version: 726-f...-disk,20240428.10,3439a13 2025-06-07T07:10:52.841Z NOTE System ROM version: M65 V920395 2025-06-07T07:10:52.845Z NOTE 2025-06-07T07:10:52.845Z START (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T07:10:52.845Z NOTE 2025-06-07T07:10:52.845Z FAIL (Issue#0897, Test #000 - HYPPO RESOURCE READ TEST FAILED) 2025-06-07T07:10:52.845Z NOTE 2025-06-07T07:10:52.845Z DONE (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T07:10:52.845Z NOTE terminating after completion of unit test
For those who haven't seen the unit test framework, it's quite nifty: The unit test running on the real hardware communicates test results back via the serial monitor interface to the m65 program, allowing it to directly report the test results.
Because I haven't implemented everything yet, the test claims to fail.
So now let's fix the implementation of our trap -- it should return from the Hypervisor by jumping to one of the return routines:
readsharedresourcetrap: stq $d681 cpq syspart_resources_area_size bcs bad_syspart_resource_sector_request ldq syspart_resources_area_start adcq $d681 ;; Ask SD card to read the sector. lda #$02 sta $d680 ;; Note that we don't wait for the request to finish -- that's ;; up to the end-user to do, so that we don't waste space here ;; in the hypervisor, where space is at an absolute premium. jmp return_from_trap_with_success
bad_syspart_resource_sector_request: ;; Return "illegal value" if trying to read beyond end of region lda #dos_errorcode_illegal_value jmp return_from_trap_with_failure
So now if we ask for an illegal sector number, we should get this illegal value error, which we know has the value 0x11 from the source.
So let's now update our test to check whether the trap is defined or not, and whether reading sector 0 results in success or an error:
// Try reading sector 0 of resource area resource_sector = 0; *(unsigned long *)0x7f0 = resource_sector;
if (!carry) { switch(PEEK(0x7f4)) { case 0xff: unit_test_fail("hyppo returned 'trap not implemented'"); break; case 0x11: // dos_errorcode_illegal_value unit_test_fail("reading resource sector 0 failed -- no resource area in system partition?"); break; default: { snprintf(message,80,"hyppo resource read failed with unknown error $%02x",PEEK(0x7f4)); unit_test_fail(message); } } unit_test_report(ISSUE_NUM, 0, TEST_DONEALL); return; }
This routine now checks whether the Hypervisor trap is implemented or not, and then whether reading sector 0 works. We need both of those to succeed, if we are going to able able to test that it works to actually read system resources.
With the fixed m65 -k option to easily load a HICKUP.M65 file containing an updated Hypervisor, I can now easily load the version that supports this new hypervisor call:
paul@bob:~/Projects/mega65/mega65-core$ make bin/HICKUP.M65 && m65 -k bin/HICKUP.M65
And then execute the test, and see the result:
paul@bob:~/Projects/mega65/mega65-tools$ make src/tests/test_897.prg && m65 -u -4 -r src/tests/test_897.prg ... 2025-06-07T07:31:16.102Z NOTE 2025-06-07T07:31:16.102Z PASS (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T07:31:16.102Z NOTE 2025-06-07T07:31:16.102Z FAIL (Issue#0897, Test #001 - READING RESOURCE SECTOR 0 FAILED -- NO RESOURCE AREA IN SYSTEM PARTITION?) 2025-06-07T07:31:16.102Z NOTE 2025-06-07T07:31:16.102Z DONE (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T07:31:16.102Z NOTE terminating after completion of unit test
And look at that --- we can see it now passes the check for the hypervisor trap being implemented (Test #000), but fails reading the first sector of the shared resource area (Test #001). This is exactly what we expect, since the SD card in this MEGA65 doesn't (yet) contain a shared resource area.
In fact, I haven't even made the utility to create them. So let's tackle that next.
Currently the MEGA65 FDISK utility is the thing that's responsible for creating the system partition. So I'll probably just tweak that to by default create a reasonable size system resource partition, perhaps 1GB or so, but maybe also have the option to select a different size.
The FDISK program is really quite old and crusty now, but it does it's job. There is an outstanding issue to totally refactor it, so while I will try to not add to the mess, I'm not going to go overboard with making this new feature overly sophisticated.
Okay, done a first cut at that. It will try to allocate half the system partition for shared resources. It also increases the target SYSPART size from 2GiB to 4GiB, thus allowing 2GiB for shared resources by default.
Now to backup my SD card, and try reformatting it!
Okay, so after a bit of procrastination, it's time to just pull the SD card, copy it all off, and put it back in, and run MEGA65 FDISK on it.
So formatting with FDISK looked promising -- it was reserving 4GB for the system partition, which is a change I made.
But then when I tried to boot with it, it was all borked up.
Basically the FAT file system seems to be messed up. But if I power on with the new HICKUP, it does see that we have $400000 sectors reserved for resources, which equates to 2GiB, so that's promising!
Okay, so why on earth is the FAT file system all messed up, then?
First thing to try: Downgrade to FDISK prior to the changes I've made, and see if it formats up fine with that.
Nope. Same thing. So probably not something that I've done.
Okay, so after a second go, including copying all my files back on, it's fine.
Who knows what went wrong.
The main thing is it is booting again.
Although, oddly it's running the intro disk, even though I don't have a default disk image to mount at boot set.
Hmmm... And now the new HICKUP is showing 0 sectors reserved for shared resources. Did I accidentally format it with the old version of FDISK? Yup --- looks like that was the case.
After I copied my files back into place, it was all fine.
And, the test to read sector 0 of the shared resource area works now:
paul@bob:~/Projects/mega65/mega65-tools$ make src/tests/test_897.prg && m65 -u -4 -r src/tests/test_897.prg ...
2025-06-07T13:30:30.590Z NOTE 2025-06-07T13:30:30.590Z START (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:30:30.590Z NOTE 2025-06-07T13:30:30.590Z PASS (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:30:30.592Z NOTE 2025-06-07T13:30:30.592Z PASS (Issue#0897, Test #001 - HYPPO RESOURCE READ TEST PASSED) 2025-06-07T13:30:30.592Z NOTE 2025-06-07T13:30:30.592Z DONE (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:30:30.592Z NOTE terminating after completion of unit test
So now I can work on making the test more complete, by doing a binary search of the address space to determine the exact size of the shared resource partition.
Like this:
resource_sector = 0xffffffffUL; for(i=31;i>=0;i--) { *(unsigned long *)0x7f0 = resource_sector;
if (test_resource_read() != 0) unit_test_fail("hyppo trap failed"); carry = PEEK(0x7f8)&1;
if (!carry) { resource_sector -= (1UL <<i); }
}
if (resource_sector==0xffffffffUL || (!resource_sector)) { unit_test_fail("failed to determine size of shared resource area"); } else { snprintf(message,80,"determined shared resource area is $%08lx sectors", resource_sector + 1); unit_test_fail(message); }
printf("Shared resource area = $%08lx sectors.\n",resource_sector+1);
And it works, correctly measuring the size!
paul@bob:~/Projects/mega65/mega65-tools$ make src/tests/test_897.prg && m65 -u -4 -r src/tests/test_897.prg /home/paul/Projects/mega65/mega65-tools/cc65/bin/cl65 --config src/tests/tests_cl65.cfg -I src/mega65-libc/cc65/include -O -o src/tests/test_897.prg --mapfile src/tests/test_897.map --listing src/tests/test_897.list src/tests/test_897_asm.s src/tests/test_897.c src/mega65-libc/cc65/libmega65.a 2025-06-07T13:41:32.282Z NOTE MEGA65 Cross-Development Tool 20250531.20-222fix-ee98359 2025-06-07T13:41:32.296Z NOTE #0: /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-07T13:41:32.296Z NOTE selecting device /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-07T13:41:32.502Z NOTE loading file 'src/tests/test_897.prg' 2025-06-07T13:41:32.691Z NOTE running 2025-06-07T13:41:32.691Z NOTE Entering unit test mode. Waiting for test results. 2025-06-07T13:41:32.691Z NOTE System model: UNKNOWN MODEL $06 2025-06-07T13:41:32.691Z NOTE System CORE version: 726-f...-disk,20240428.10,3439a13 2025-06-07T13:41:32.691Z NOTE System ROM version: M65 V920395 2025-06-07T13:41:32.694Z NOTE 2025-06-07T13:41:32.694Z START (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:41:32.694Z NOTE 2025-06-07T13:41:32.694Z PASS (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:41:32.694Z NOTE 2025-06-07T13:41:32.694Z PASS (Issue#0897, Test #001 - HYPPO RESOURCE READ SECTOR 0 TEST PASSED) 2025-06-07T13:41:32.695Z NOTE 2025-06-07T13:41:32.695Z PASS (Issue#0897, Test #002 - DETERMINED SHARED RESOURCE AREA IS $00400000 SECTORS) 2025-06-07T13:41:32.696Z NOTE 2025-06-07T13:41:32.696Z DONE (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-07T13:41:32.696Z NOTE terminating after completion of unit test
Okay, so now the last thing to test is that it actually causes the SD card to read the sector.
First, do the SD card registers get the correct sector number written into them? I'll test this by just making sure that they don't end up with all zeroes. Anyway, nope, they don't. Found and fixed -- the bug was in the hypervisor code (the interested reader can look at the code snippet for the hypervisor trap above and see if they can spot it --- leave a comment if you find it :)
So then the next step is to check that the SD sector buffer actually changes contents. That worked fine, without further changes (although I had to make sure the SD card was not busy, before making the HYPPO call, because requesting a sector read while the SD card is busy causes it to lock up).
Next up, then, is to come up with a simple file system for the shared resource area. It needs to be super simple to parse. I'm thinking it could be as simple as the first sector containing version information and the number of resources in the area, followed by one sector per file, that contains the name of the file/resource, and the start sector within the resource area, and the length of the resource in both sectors and bytes. After the final resource directory sector, we have a completely empty sector, so that a program scanning it can just look for that empty sector and know to stop, without even having to track the resource count, in case that's easier to implement in a given situation.
So let's make a utility that can build the shared resource area from a set of files and their names. Since it's just boring boiler-plate kind of code, I used ChatGPT to do it faster. Not sure if it actually was any faster to write. But we can now do things like this:
SHA1 Bytes Check Name -------------------------------------------------------------------------------- 40ce6cfcaaf14d11a8c3ffa71fd4726087613eee 4242 MATCH mega65-screen-000000.png 3b31809d69f814d2173484855072333db65190cb 6436 MATCH README.md paul@bob:~/Projects/mega65/mega65-tools$
Next step is to make a tool to extract or insert a shared resource area onto/off of an actual SD card. My plan here is to make the existing tool I've made above to work out if it's writing to an SD card, in which case, it will find the SYSPART in there, and write the resources directly into there.
Okay, so I think I have writing the resources to SD card working, as well as updating shresls so that it can read and check them on a real disk image, too:
paul@bob:~/Projects/mega65/mega65-tools$ rm bin/shresls ; make bin/shres bin/shresls && sudo bin/shres /dev/sdb mega65-screen-000000.png,31 README.md,23 make: 'bin/shres' is up to date. gcc -Wall -g -std=gnu99 `pkg-config --cflags-only-I --libs-only-L libusb-1.0 libpng` -mno-sse3 -march=x86-64 -o bin/shresls src/tools/shresls.c -lssl -lcrypto INFO: Attempting to open shared resource file or disk image '/dev/sdb' INFO: Found MEGA65 SYSPART at sector 0x014d7ffe INFO: Found MEGA65 SYSPART shared resource area of 2048 MiB. DEBUG: area_start=0x00029b0ffc00, area_length=0x80000000 Resource Table (2 entries): Idx Start Sectors Bytes Flags Name ---- ------- -------- ---------- ---------- ------------------------- 0 4 9 4242 0x80000000 mega65-screen-000000.png 1 13 13 6436 0x00800000 README.md
paul@bob:~/Projects/mega65/mega65-tools$ make bin/shresls && sudo bin/shresls /dev/sdb gcc -Wall -g -std=gnu99 `pkg-config --cflags-only-I --libs-only-L libusb-1.0 libpng` -mno-sse3 -march=x86-64 -o bin/shresls src/tools/shresls.c -lssl -lcrypto INFO: Attempting to open shared resource file or disk image '/dev/sdb' INFO: Found MEGA65 SYSPART at sector 0x014d7ffe INFO: Found MEGA65 SYSPART shared resource area of 2048 MiB at sector 2048 of SYSPART. MEGA65 Shared Resource File: /dev/sdb Declared resources: 2
SHA1 Bytes Check Name -------------------------------------------------------------------------------- 40ce6cfcaaf14d11a8c3ffa71fd4726087613eee 4242 MATCH mega65-screen-000000.png 3b31809d69f814d2173484855072333db65190cb 6436 MATCH README.md
So now I just need to make some tests for scanning the shared resource list, and trying to read them. Actually, all we probably need is to check for the magic string in the shared resource area. This is because for that to be found, it means that the SD card access is working, as well as the sector address calculation.
And it works!
mega65-tools$ make src/tests/test_897.prg && m65 -4 -r -u src/tests/test_897.prg make: 'src/tests/test_897.prg' is up to date. 2025-06-13T07:15:45.034Z NOTE MEGA65 Cross-Development Tool 20250531.20-222fix-ee98359 2025-06-13T07:15:45.048Z NOTE #0: /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-13T07:15:45.048Z NOTE selecting device /dev/ttyUSB0 (Arrow; 0403:6010; undefined; 03636093) 2025-06-13T07:15:45.254Z NOTE loading file 'src/tests/test_897.prg' 2025-06-13T07:15:45.447Z NOTE running 2025-06-13T07:15:45.447Z NOTE Entering unit test mode. Waiting for test results. 2025-06-13T07:15:45.447Z NOTE System model: UNKNOWN MODEL $06 2025-06-13T07:15:45.447Z NOTE System CORE version: 726-f...-disk,20240428.10,3439a13 2025-06-13T07:15:45.447Z NOTE System ROM version: M65 V920395 2025-06-13T07:15:45.451Z NOTE 2025-06-13T07:15:45.451Z START (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-13T07:15:45.451Z NOTE 2025-06-13T07:15:45.451Z PASS (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-13T07:15:45.451Z NOTE 2025-06-13T07:15:45.451Z PASS (Issue#0897, Test #001 - HYPPO RESOURCE READ SECTOR 0 TEST PASSED) 2025-06-13T07:15:45.452Z NOTE 2025-06-13T07:15:45.452Z PASS (Issue#0897, Test #002 - DETERMINED SHARED RESOURCE AREA IS $00400000 SECTORS) 2025-06-13T07:15:45.455Z NOTE 2025-06-13T07:15:45.455Z PASS (Issue#0897, Test #003 - SD CARD BUSY CLEARED ON RESET) 2025-06-13T07:15:45.456Z NOTE 2025-06-13T07:15:45.456Z PASS (Issue#0897, Test #004 - SD CARD REGISTERS WRITTEN TO) 2025-06-13T07:15:45.456Z NOTE 2025-06-13T07:15:45.456Z PASS (Issue#0897, Test #005 - SD CARD BUSY CLEARED) 2025-06-13T07:15:45.456Z NOTE 2025-06-13T07:15:45.456Z PASS (Issue#0897, Test #006 - SD CARD BUFFER CONTENTS CHANGE) 2025-06-13T07:15:45.456Z NOTE 2025-06-13T07:15:45.456Z PASS (Issue#0897, Test #007 - READ MAGIC STRING FROM SHARED RESOURCE AREA) 2025-06-13T07:15:45.457Z NOTE 2025-06-13T07:15:45.457Z DONE (Issue#0897, Test #000 - HYPPO SYSTEM RESOURCE ACCESS) 2025-06-13T07:15:45.457Z NOTE terminating after completion of unit test
Okay, so now I have all the ingredients here for this to work -- HYPPO support for the shared resource area, patched MEGA65 FDISK to create the section in the system partition, linux-based tools to populate and interrogate the shared rescource area quickly and easily, and unit tests that provide proof-of-concept for accessing this area, and checking that the loaded HYPPO has support for it.
So I'm going to leave it at that for now, and I'll document writing a library to open and read from shared resources in a separate blog post.
I'll also pursue getting the HYPPO changes rolled into mega65-core development branch, and updated FDISK into mega65-fdisk development branch.
The next module on the list is something that can do BlueTooth to talk to a headset. For security reasons, I'd prefer something that is BlueTooth only, if I can find it. The problem is that the ESP32 is probably otherwise the obvious choice, but I'm not totally sure that it can be trusted. But then, there is lots of alternative firmware available for it, and at the end of the day, if you are using wireless to talk to a headset, then you have already compromised your own security.
But let's look at the options...
1. Requirements Analysis
The requirements relating to this module are:
Requirement 2.1.1, Bluetooth host module that can connect to headsets and other audio devices. Must support UART or I2C control, and audio interfaces that we can easily implement from the FPGA, such as I2S audio.
Desirable 2.1.2, Bluetooth host module support for keyboards and other HID peripherals.
Note that 2.1.2 was previously listed as a requirement, but it is not required in the milestones, and so I have allowed it to be relaxed to a desirable. I'll document this in the repository.
2. Key Component Selection
I do have a BM63 eval kit laying around, which I had originally intended for this use-case, but I believe it is only able to act as a sink (headset/speaker), not a source (phone). But chatgpt is trying to tell me that it can do both -- and I fear it is hallucinating this, as much as I'd like to believe that it could do it.
This all feels vaguely familiar, like I explored this in the past. I'm staring at the BM63 eval kit I have here, and wondering if I didn't get a BM83 kit as well at some point. It would be nice if I did, as they are A$500+ each.
Ah, it _is_ a BM83 kit -- its just my aging eyes in the dim light here in the middle of the night! Hurrah!
So I can immediately start setting that up, and trying to make it pair with a BT headset, and see how it goes.
I'm starting to read through the documentation, so that I can do some basic testing of this module to make sure it works -- I want it to pair with a headset and pretend to start sending audio to it, and also be able to pretend to start a voice call. I should be able to hear the difference in noise in the ear piece of the old Bose QC35s I have here to tell the difference.
The BM83 supports an "AT command set". Now, this is _not_ a modem-style Hayes-AT command set, but rather an Audio Transceiver Command Set, which is binary in format.
But looking at the commands, it looks like everything I should need is there. Time for sleep now, but I'll start setting up the board in the morning.
Okay, so now I'm reading my way through the BM83 Evaluation Board User's Guide, with the intention of powering it up and talking to it via UART. That board has a PIC32 microcontroller on it, which I don't want to use, so I may need to figure out which of the many switches need to be set correctly to run the BM83 in stand-alone UART controlled mode.
Following the instructions in section 4, I can put it into pairing mode and make it pair with my phone -- thus confirming that the module is superficially working, which is great. Except that after pairing, the phone refuses to actually connect with it. But it's a cheap android phone, so it could be that. Also it shows up on my phone under "Other devices" which might mean that it doesn't recognise the set of profiles its offering. After some more pressing the MFB button in semi-random ways, I have gotten it to pair, and it is showing support for HD-Audio AAC, Telephone calls and audio media, which is great. The buttons to control the volume etc, however, are still not causing anything to happen. But I don't care too much about that right now, as I intend for it to be used primarily in the reverse direction: as the phone, rather than the headset.
Switching SW300 on lets me seemingly switch the BM83 eval board into "test mode" where I can talk directly to it via the USB UART.
Then I can use the isupdate.exe tool from Microchip to talk to it.
I had a frustrating half hour where I could make it connect, and it would even identify it as BT5511_002 in a little box, but then fail to connect. After some googling around, I found this post, with this sage piece of advice:
Put the TTY into raw mode:
stty -F /dev/ttyACM0 raw
With that, I was able to flash the main firmware, the DSP firmware and the application firmware (you have to do them all -- there is a way to "rehex" them to make a single file for convenience, but I didn't do that) with SW300 set to on, and then turn it off again after flashing everything.
This also confirms that SW300 should be off for normal operation. I did find that if I press RESET_N and then long-press (~1 sec) the MFB button, that the BM83 does output something that looks like the kind of AA + checksum packet format we expect:
AA 00 03 30 00 00 CD AA 00 03 2D 05 01 CA
So let's try to understand how those should be decoded.
Except I can't even get it to do that reliably now. I suspect the PIC32 MCU on the eval board might still be doing things and getting in the way.
Let's try to get to the bottom of this.
We want "host mode", where the BM83 expects an MCU to be connected via UART. That's good, because we have the host firmware loaded.
But the eval board will be expecting to have the PIC32 act as the MCU in this case. We need to disable that. I'm finding the documenation for this evaluation board quite frustrating, to be honest. For example, there is a big fat JP303 connector with 40 signals brought out onto it -- but it isn't even mentioned in the user's guide. And it's not like its a different PCB revision that lacks it -- you can see the big 2x20 0.1" header at the top of the PCB in this screenshot, and how there is carefully nothing pointing at it. JP303, which is silkscreened next to it isn't event mentioned once in the manual.
Fortunately despite this seemingly intentional ignorance of it, it has various signals identified on it in silkscreen. Those of interest to us are:
HCI_RXD
HCI_TXD
Indeed HCI_TXD follows the UART TX line from the USB UART adaptor. So my bits are getting that far at least. And HCI_RXD really seems to be connected to the BM83, because I see those responses that I've been seeing when pressing the MFB ("Multi-Function Button", by the way). The MFB pin is also routed out onto this JP303.
A bit more digging indicates that S400 selects bewteen onboard PIC32 and an external microcontroller. It was already set to external, which is what I want. So most probably the PIC32 isn't actually being a problem.
So why doesn't the firmware talk to me in any more sensible way?
The first assumption is that the "application" firmware that I loaded on isn't the right one, or doesn't support the UART packetised commands.
So I have perhaps found a clue on this page. It talks about running the config utility to enable the BM83's UART. Maybe RX on the UART has to be enabled? But I find it a bit odd that the UART can send, but not receive.
The datasheet does seem to indicate that this GUI config tool is required to configure host mode. The problem is that that is windows-only software, and I haven't figured how to get it running under Wine yet. Or rather, I can run it, but it so far makes no sense at all. It doesn't seem to communicate with the module but rather allows loading and saving of configurations, without a great deal of clarity as to how to actually drive it.
To add to the fun, there is corrupted copy of the documentation for this config tool in the SDK:
Ok, so the tool is quite a bit newer than in the documentation. And the order of the options has changed.
But broadly speaking, it can be followed, and when you save it, it writes a .HEX file that can then be flashed onto the device using the isupdate.exe program.
After a lot of poking around, it looks like the MSPKv2_1.03.0406_SPP firmware is missing many of the UART commands that we'd need. But potentially they are in the v1.3 era. But finding that old firmware is proving non-trivial. It's not helping that the Microchip website is under maintenance for a few hours right now.
Actually, it looks like the firmware I downloaded fresh is not the latest according to this post. There should be a 2v1.3.5 instad of the 2v1.2.4 that I have. Now to try to find a link where I can download it...
I've since posted a question on the Microchip forum and emailed my local FAE for their distributor to see if I can get some help with this, as I'm just spinning my wheels, when I have much more that I need to get on with.
I'm going to work on the assumption that there will be some solution to this problem at some point, and since the module does pair with things, that it fundamentally works. In short, I can get on with designing a module to carry the BM83.
3. Schematic Layout
So for this one, we really just need to host the BM83, and work out which of its 50 pins we actually care about. So let's take a look:
Okay, so that's our pin out. Let's now work out which ones we definitely need, which we definitely don't need, and if there are any we aren't sure about.
BM83 Module Pin Description
Pin Number
Pin Name
Required?
Description
1
DR1
YES (I)
I2S interface: digital left/right data
2
RFS1
YES (I/O)
I2S interface: digital left/right clock
3
SCLK1
YES (I/O)
I2S interface: bit clock
4
DT1
YES (O)
I2S interface: digital left/right data
5
MCLK1
YES (O)
I2S interface: master clock
6
AOHPR
NO
R-channel analog headphone output
7
AOHPM
NO
Headphone common mode output/sense input
8
AOHPL
NO
L-channel analog headphone output
9
MICN2
NO
MIC 2 mono differential analog negative input
10
MICP2
NO
MIC 2 mono differential analog positive input
11
AIR
NO
R-channel single-ended analog input
12
AIL
NO
L-channel single-ended analog input
13
MICN1
NO
MIC 1 mono differential analog negative input
14
MICP1
NO
MIC 1 mono differential analog positive input
15
MICBIAS
NO
Electric microphone biasing voltage
16
GND
YES (P)
Ground reference
17
DMIC_CLK
NO
Digital MIC clock output
18
DMIC1_R
NO
Digital MIC right input
19
DMIC1_L
NO
Digital MIC left input
20
P3_2
MAYBE (I/O)
GPIO (default: AUX_IN DETECT)
21
P2_6
MAYBE (I/O)
General purpose I/O port
22
ADAP_IN
NO (P)
5V adapter input / USB DFU / battery charge
23
BAT_IN
YES (P)
Power supply input (3.2V–4.2V)
24
SYS_PWR
NO
System power output (internal use only)
25
VDD_IO
YES (P)
I/O power supply (internal use)
26
PWR (MFB)
YES (I)
Multi-function push button
27
SK1_AMB_DET
MAYBE (I)
Temperature sense channel 1
28
SK2_KEY_AD
MAYBE (I)
Temperature sense channel 2
29
P8_6 / UART_RXD
YES (I/O)
GPIO / UART RX data
30
P8_5 / UART_TXD
YES (I/O)
GPIO / UART TX data
31
P3_4 / UART_RTS
YES (I/O)
GPIO / UART RTS / Mode select
32
LED1
MAYBE (I)
LED driver 1
33
P0_2
MAYBE (I/O)
GPIO (default: play/pause)
34
LED2
MAYBE (I)
LED driver 2
35
P0_6
MAYBE (I/O)
GPIO
36
DM
NO (I/O)
USB D-
37
DP
NO (I/O)
USB D+
38
P0_3
MAYBE (I/O)
GPIO (default: reverse)
39
P2_7
MAYBE (I/O)
GPIO (default: volume up)
40
P0_5
MAYBE (I/O)
GPIO (default: volume down)
41
P1_6 / PWM1
MAYBE (I/O)
GPIO / PWM1 output
42
P2_3
MAYBE (I/O)
GPIO
43
RST_N
YES (I)
System reset (active-low)
44
P0_1
MAYBE (I/O)
GPIO (default: forward)
45
P0_7
MAYBE (I/O)
GPIO
46
P1_2 / TDI_CPU / SCL
MAYBE (I/O)
GPIO / Debug / I2C SCL
47
P1_3 / TCK_CPU / SDA
MAYBE (I/O)
GPIO / Debug / I2C SDA
48
P3_7 / UART_CTS
MAYBE (I/O)
GPIO / UART CTS
49
P0_0 / UART_TX_IND
YES (I/O)
GPIO / Codec reset (Embedded) / TX_IND (Host)
50
GND
YES (P)
Ground reference
So that's 15 yeses and 18 maybes. That would make 33 total, if we include everything that we could feasibly want (given we are only using this to send and receive I2S audio. I am a bit concerned about whether the BM83 can convey bidirectional audio from a call via it's digital I2S interface, but we can live without that for this initial version, as the microphone array on the MEGAphone unit itself will be fine to use for many use-cases. For true hands-free operation, e.g., a headset in a bike helmet, like I use from time to time, we'd need it though. But as said, we will defer that for now.
Back to our 33 pins, that means that we need at least a 2x17 module, which would be 1.7 inches long. The BM33 is 22mm from the bottom to where the PCB antenna starts, i.e., ~0.9 inches. So if we want all those extra pins, we will end up using more board realestate than we'd like. So let's prune out the bluetooth fuction buttons, e.g., reverse, volume up, down, forward, play (5 pins), the temperature sensor (2 pins), the stray GPIOs (5 pins), and that leaves us with 21 pins, which would fit just about perfectly -- and still leave us with the most likely pins we'd need. The updated table would look like this:
BM83 Module Pin Description
Pin Number
Pin Name
Required?
Description
1
DR1
YES (I)
I2S interface: digital left/right data
2
RFS1
YES (I/O)
I2S interface: digital left/right clock
3
SCLK1
YES (I/O)
I2S interface: bit clock
4
DT1
YES (O)
I2S interface: digital left/right data
5
MCLK1
YES (O)
I2S interface: master clock
6
AOHPR
NO
R-channel analog headphone output
7
AOHPM
NO
Headphone common mode output/sense input
8
AOHPL
NO
L-channel analog headphone output
9
MICN2
NO
MIC 2 mono differential analog negative input
10
MICP2
NO
MIC 2 mono differential analog positive input
11
AIR
NO
R-channel single-ended analog input
12
AIL
NO
L-channel single-ended analog input
13
MICN1
NO
MIC 1 mono differential analog negative input
14
MICP1
NO
MIC 1 mono differential analog positive input
15
MICBIAS
NO
Electric microphone biasing voltage
16
GND
YES (P)
Ground reference
17
DMIC_CLK
NO
Digital MIC clock output
18
DMIC1_R
NO
Digital MIC right input
19
DMIC1_L
NO
Digital MIC left input
20
P3_2
NO
GPIO (default: AUX_IN DETECT)
21
P2_6
NO
General purpose I/O port
22
ADAP_IN
NO (P)
5V adapter input / USB DFU / battery charge
23
BAT_IN
YES (P)
Power supply input (3.2V–4.2V)
24
SYS_PWR
NO
System power output (internal use only)
25
VDD_IO
YES (P)
I/O power supply (internal use)
26
PWR (MFB)
YES (I)
Multi-function push button
27
SK1_AMB_DET
NO
Temperature sense channel 1
28
SK2_KEY_AD
NO
Temperature sense channel 2
29
P8_6 / UART_RXD
YES (I/O)
GPIO / UART RX data
30
P8_5 / UART_TXD
YES (I/O)
GPIO / UART TX data
31
P3_4 / UART_RTS
YES (I/O)
GPIO / UART RTS / Mode select
32
LED1
MAYBE (I)
LED driver 1
33
P0_2
NO
GPIO (default: play/pause)
34
LED2
MAYBE (I)
LED driver 2
35
P0_6
NO
GPIO
36
DM
NO (I/O)
USB D-
37
DP
NO (I/O)
USB D+
38
P0_3
NO
GPIO (default: reverse)
39
P2_7
NO
GPIO (default: volume up)
40
P0_5
NO
GPIO (default: volume down)
41
P1_6 / PWM1
NO
GPIO / PWM1 output
42
P2_3
NO
GPIO
43
RST_N
YES (I)
System reset (active-low)
44
P0_1
NO
GPIO (default: forward)
45
P0_7
NO
GPIO
46
P1_2 / TDI_CPU / SCL
MAYBE (I/O)
GPIO / Debug / I2C SCL
47
P1_3 / TCK_CPU / SDA
MAYBE (I/O)
GPIO / Debug / I2C SDA
48
P3_7 / UART_CTS
MAYBE (I/O)
GPIO / UART CTS
49
P0_0 / UART_TX_IND
YES (I/O)
GPIO / Codec reset (Embedded) / TX_IND (Host)
50
GND
YES (P)
Ground reference
So let's filter out all the NOs and see how it looks:
BM83 Module Pin Description
Pin Number
Pin Name
Required?
Description
1
DR1
YES (I)
I2S interface: digital left/right data
2
RFS1
YES (I/O)
I2S interface: digital left/right clock
3
SCLK1
YES (I/O)
I2S interface: bit clock
4
DT1
YES (O)
I2S interface: digital left/right data
5
MCLK1
YES (O)
I2S interface: master clock
16
GND
YES (P)
Ground reference
23
BAT_IN
YES (P)
Power supply input (3.2V–4.2V)
25
VDD_IO
YES (P)
I/O power supply (internal use)
26
PWR (MFB)
YES (I)
Multi-function push button
27
SK1_AMB_DET
MAYBE (I)
Temperature sense channel 1
28
SK2_KEY_AD
MAYBE (I)
Temperature sense channel 2
29
P8_6 / UART_RXD
YES (I/O)
GPIO / UART RX data
30
P8_5 / UART_TXD
YES (I/O)
GPIO / UART TX data
31
P3_4 / UART_RTS
YES (I/O)
GPIO / UART RTS / Mode select
32
LED1
MAYBE (I)
LED driver 1
34
LED2
MAYBE (I)
LED driver 2
43
RST_N
YES (I)
System reset (active-low)
46
P1_2 / TDI_CPU / SCL
MAYBE (I/O)
GPIO / Debug / I2C SCL
47
P1_3 / TCK_CPU / SDA
MAYBE (I/O)
GPIO / Debug / I2C SDA
48
P3_7 / UART_CTS
MAYBE (I/O)
GPIO / UART CTS
49
P0_0 / UART_TX_IND
YES (I/O)
GPIO / Codec reset (Embedded) / TX_IND (Host)
50
GND
YES (P)
Ground reference
Okay, so I relented, and added the temperature sensor back in, because I think it might be good to know the internal temperature of the unit, and I'd miscounted pins, so this still leaves us at just 22 pins, which feels acceptable.
Ah, there are also two GND pads, so that saves us a pin. Also, the VCC_IO pin isn't needed either, apparently, according to the footprint that I'm using. The footprint is annoying in that it has a keep-out zone for copper surrounding the GND pads on the rear of the PCB! I've just modified that so that it allows vias, so that I can still connect those pads to GND.
The end result is a very simple schematic:
Note that we feed 3.3V to the modules VBAT input. This is so that we can still switch power on and off to this module, rather than running it directly from the battery.
4. PCB Layout
The PCB layout is also trivial, with the pin assignments on the module being mapped so that all routing happens on the front, except for the power and ground which are routed on the rear.
5. Requirements Cross-Check
Requirement 2.1.1,
Bluetooth host module that can connect to headsets and other audio
devices. Must support UART or I2C control, and audio interfaces that we
can easily implement from the FPGA, such as I2S audio.
Met -- in that we have the Bluetooth module, and the necessary interfaces for I2C and I2S and UART.
Desirable 2.1.2, Bluetooth host module support for keyboards and other HID peripherals.
Met if and only if the BM83 can be configured to talk to HID devices. This remains unknown, but as this is only a desirable rather than a requirement, we are fine.
I've been busy working on the schematic and PCB design for a bunch of the modules for the MEGAphone lately. The first batch of those I had sent off for fabrication by PCBWay, and they have finally arrived. So let's have a look at what we have:
Well, that's a bit hard to see what's going on, so let's look at each in turn. My lighting here isn't that great right now, but it should be enough that we can identify each PCB from it's packing label.
First we have the audio jack PCB:
Then the high-efficiency DC:DC converter:
The Audio Codec board:
SIM and SD-Card carrier:
Cellular modem carrier:
MPPT Battery Charger board:
Low-current DC:DC converter module:
And finally, the MEMS microphone carrier board.
So let's look at where this gets us to in terms of the milestones:
Milestone 2.3.4 Battery management and energy harvesting module to
allow USB-C, integrated solar and external 12/24V vehicle battery power
sources, and efficient management of the integrated rechareable battery: Fabrication -- done -- see photos above
Milestone 2.5.4 Internal microphone and speaker module: Fabrication -- done -- see photos above
Let's see how this looks on the block-diagram, to get a sense of the overall progress represented by this work:
Not bad -- and as we can see, some of these modules are reused in multiple places, so we have knocked off most of the orange boxes, too.
The main difference between what was planned and how I have progressed is that I've tended to further break down some of the modules, to make their designs simpler.
From here, I'll decide whether the next step is to assemble these modules, or whether I'll do the PCB design for more of the remaining modules on the schedule. Probably a mix of the two. Certainly I can prepare the bill of materials for each of the modules I've already designed, and order those parts ready for assembly, since they will take a week or so to arrive, in any case.
This post has turned into a bit of an epic, as it turned out to have a couple of modules (solar MPPT tracker and prioritised power input selection and over-voltage and reverse-polarity protection) that took quite a bit more effort to get right.
So, we have a DC:DC regulator. Now we need a way to get energy into the battery. This module has a few more requirements, because of the different energy sources we intend to support.
1. Requirements Analysis
There are also a few requirements that we have to keep in mind, even though this module isn't directly responsible for satisfying them (in italics):
Requirement 1.1.2.3: Proof-of-concept charger module based on a given USB-C power delivery module (no need for data). Requirement 1.1.2.4: Power isolation mechanism to allow conflict-free operation of the USB-C and alternative power inputs to the MEGAphone (back-flow and over-voltage protection, for example) Requirement 1.1.1: Integrated LiFePO4 or other similarly safe and robust chemistry battery cell. Requirement 1.1.2.1: Integrated battery capacity of at least 30Wh. Requirement 1.2.2.1: Means of accepting (including connector) dirty DC power between ~9V and ~40V using a MPPT to maximise efficiency with a solar panel. Requirement 1.2.3.1: Means of accepting DC power from integrated solar panel at input voltages of 2V to 20V with a MPPT.
So, we have to accept power from USB-C, solar, and "dirty DC" -- which itself might be a solar panel. I'm going to break this down into several modules, to make my life easier.
Requirement 1.2.3.1 is a bit annoying, because the 2V I have specified is not commonly supported. We can resolve this problem by using readily available panels that have a nominal voltage of 6V, e.g.:
Those 55x70mm panels are 0.5W, and have a nominal voltage of around 6V. It should be easy enough to tile four of those on the rear, to give about 2W solar charging capacity on-board. Those could be put in two parallel strings of 2 panels to yield ~12V.
But remember that we also want to support external panels that might have higher voltages. Ideally, we'd allow upto ~40V, so that this module could also accept power from automotive sources.
So let's now look at the options for MPPT ICs. We don't need high current output, as the battery is only 30Wh, and charging in 1 hour would require 10A, so probably "a few Amps" would be tolerable -- but if we can, I'd like to support fast charging in an hour or two if a high-current source is available.
These various requirements make the design of this module more complicated. Fortunately, I'm not the first (or last) person to want to do this kind of thing.
For example, there is this video and github project for exactly the kind of thing I want:
It's even been designed using KiCad, which means I can adapt it easily. I hadn't planned on putting a little display to show the battery charge state, but I can always leave it off -- or if space and topology of the boards allow it, I could still include it.
The unit itself is fairly small. It does use an ESP32, which is possibly a bit of an over-kill, but if that's the cost of making this module more quickly, then it's probably worth it, as this is one of the most complex modules that I'll need to design.
I'd likely still redesign the board to fit the castellated module system, but that would be the only change I'd make, to minimise risk, I think. Oh, except that I'll also need to change the battery charger part of it to handle a single-cell LiFePO4.
That project uses a BQ24650, which does apparently support LiFePO4 cells. The battery charge voltage is set through a voltage divider:
This makes it super simple to change the output voltage of the charge circuit. The question is what voltage to charge to.
ChatGPT claims that 3.6V should be ok, but may lead to damage over time if that voltage is held after the battery fully charges. 3.45V -- 3.55V on the other hand should allow the battery life to be much greater, but at the loss of ~10% of the battery capacity.
As we are using a 10Ah LiFePO4 cell with a capacity of ~36Wh, that would still leave us >30Wh, which is above our requirement, so it sounds like a good trade-off.
Looking at the schematic and ESP32 firmware, it also looks like it is possible for the ESP32 to be able to turn the charger off, and also to measure the battery voltage. Thus I could in a later version update the ESP32 firmware to do that, and then allow charging to the 3.6V 100% level. But we don't need to for now.
So back to the R2 & R1 calculation, we want V_BAT=3.5V, so:
3.5V = 2.1V x ( 1 + R2/R1)
Thus 1+ R2/R1 = 3.5V / 2.5V = 1.4.
Keeping R1 at 100K in the module, we just need to change R2 to 140K. That's a standard size, so no problem there.
Otherwise the only requirement adjustment forced on us by this module is the maximum input voltage is ~28V, rather than 40V. This means that while we could easily accept 12V car power and 20V USB-C power, we couldn't safely accept 24V truck power, as the voltage on trucks typically peaks around 30V. It's pretty close, and I suspect that it would work in practice. So we'll live with it.
2. Requirement Variations
So let's summarise our requirements relaxations that we have applied:
Relaxation of Requirement 1.2.2.1: Maximum input voltage reduced from 40V to ~28V. Impact: (1) 24V truck systems may peak out of range when alternator charging. (2) Some larger solar panels may be out of spec. However, given the ~30Wh battery size, this should not be a problem. Impact acceptable.
Relaxation of Requirement 1.2.3.1: Minimum solar panel input voltage increased from 2V to 6V. Impact: Some very small panels, especially those consisting of few cells in series, no longer in specification. However, small panels (eg 55x70mm) are readily available with 6V MP voltages. Thus the impact is acceptable.
3. Solar MPPT / Dirty DC Input & Battery Charger Module
Those changes to the requirements have been recorded in the repository, so now we are good to go with adapting the module to fit one of our modules.
Let's list our planned changes:
1. Change board outline to appropriate sized castellated module.
2. Add castellated pads for VBAT and GND_BAT. This will be how battery power reaches the rest of the system.
3. Route ESP32 UART RX and TX to castellated pads to allow querying and control of the module in the future (e.g., to ask battery charge level, and charge system status).
4. Route 3.3V rail from charge source (not from the battery) to castellated pads, so that any galvanic isolation for the UART can be powered from the charge source, so that when there is no charge source, the MPPT charger is fully depowered, so that it doesn't discharge the battery.
5. Route LOAD_ENABLE and VCC_LOAD pins to castellated pads, to allow for a switchable load at VBAT. Not sure that we will need to make use of these, but it seems silly to not make them available.
6. Route charge source voltage to castellated pads instead of a terminal block.
I think that's it -- It really is a testament to Bytesizedengineering in their design of this module, that it has pretty much everything that I need (and then some). I was hoping to find something like this, as it is saving a lot of time and trial-and-error.
We will, however, also need another module to address Requirement 1.1.2.4. It looks like the LM74700-Q1 with an external MOSFET is a good approach here, as they have losses of only ~40mW on input, and disconnect the power source when current tries to flow in the reverse direction. We'll need one for each input, so three replicates of that module, which is fine -- it's another advantage of the module based approach.
Okay, so that's everything we need to know right now, and I can dive in to adapting the MPPT tracker... except that I need to update KiCad first, because it was created using a newer version.
A week or so has passed, and a few things have happened in the meantime:
1. Zach, the author of the module I am looking to adapt kindly replied to my email asking about licensing of his design, and has indicated that it's GPLv3, which is good, because that's what I'm using for this repo as a whole.
2. Zach also indicated that it has some bugs and problems. I'm hoping to learn a bit more about that, so that I can update the module if necessary. But for now, I'm working on the assumption that it works well enough for what I need.
So now I can easily play around with what size module I think I need, and if it turns out wrong, or has the wrong number of pins, I can easily generate a new one.
Okay, so I'm making some progress, and currently trying to work out how the power supply to the various ICs on this module work. There are several, and they need 3.3V, but I can't yet see where that comes from. Ah, it comes from the AP63203 that is powered from the input source, i.e., the ESP32 and friends will only power on if there is a DC source. So that's good.
Then as I've dug deeper, the ACS712 current sensors seem to need VCC=5V,
not the 3.3V they have been given in the design. But it seems that people have found them to work with VCC=3.3V. Resolving this is tricky, as it will output a signal of up to VCC, so giving it 5V will mean I need to add a level converter. Conversely, feeding it on VBAT will mean that it can go below 3.3V. But as it's only a current sensor for power input and charge output, maybe that's tolerable: It will only be able to mis-behave if VBAT is too low, but we can measure VBAT directly on the ESP32, so if VBAT is low, but the ESP32 is on, we can just make sure the ESP32 firmware tries to charge the battery in that case.
The ACS712 might be interesting to have elsewhere in the system, because it would let us measure the current draw from the battery to monitor power better, but that can wait.
So back to this module: The ACS712 can be powered from VBAT in the first instance, and if it proves to be a problem, I'll deal with it. I'll route the VCC lines for the two ACS712s to a pad on the module, so that this can be done off-module if required.
The ACS712 comes in 5A, 20A and 30A sensing versions. With VCC=5V, the 5A version will reach 3.3V on its output only at >= 4A, while for the 20A version it will be > ~7A, or for the 30A version > ~12A. So one option is that we just make sure that the thing never gets more than that level of current fed into it. But that's hard to force. But we could just put a 3.3V Zener diode to clamp the voltage at 3.3V, to make sure that the ESP32 doesn't get fried.
For the battery charge current side, we can set the charge current limit using RSR1 according to the BQ24650 charge controller's datasheet. If we limit it to 4A, then we will keep the ACS712 sensing the battery charge current <= 3.3V, which avoids the need to clamp the output of that ACS712.
Now, because the LiFePO4 battery is ~3.6V max, and assuming that the BQ24650 isn't horribly inefficient, and that it requires an input voltage on the panel/DC source >5V, then the current flowing through the input ACS712 must be lower than that of the output one. And if the output one is safe at 4A, then the input one will be, too.
So all up, this means that we can charge the battery at up to 4A, for a total charge time of ~2.5 hours in the ideal case, without needing to clamp the outputs of the ACS712. That would correspond to a charge rate of 4Ax3.5V = 14W, which feels good enough.
So the question is whether we need to provide a 5V supply for those ACS712's, or just take the chance with 3.3V. I could add an AP63205 to provide the 5V that they need. Hmm... Let's just follow the Wisdom of the Internet (tm) and run them on 3.3V, but provide a separate pad for their VCC, so that by default we can connect them together, but can separate them and provide an external 5V source if required.
I've spent the last few hours mashing everything around to fit in the module, and then dealing with design rule violations in KiCad, to make sure that I haven't messed anything up. There were a lot related to silkscreen stuff, a few of which are hard to resolve due to components with pads that intersect their own silkscreen elements. But there are less than a dozen of those now, so I can easily see if there are real problems.
Here is how it looks now:
The large oval and hole-throughs is for the ESP32 module that it uses. This also means that we need to use the cut-out to allow the hole-through pins of the ESP32 to clear the carrier board PCB. In other words, the ESP32 will sit up above the carrier board, rather than just having components in the cut-out behind the module (though it has a few components on the reverse side, too).
The presence of components on both sides of the board will make it a bit trickier for me to assemble. I'll likely reflow the main component side using a hot plate, and then hand-solder the small number of components on the rear-side. If I got a fancy reflow oven I could use high-temperature solder for the components on the main side, reflow that, and then flip it over and use a lower-temperature solder for the reverse side and reflow that below the melting temperature of the solder on the main component side, so that nothing falls off.
But before I am ready to send the board off for fabrication, I want to do my usual thing of adding component values and part names to as many of the SMD components as possible, to make it easier to field repair or scavenge for parts in the future.
In the process of that, I have found that there was no part number identified for the dual-MOSFET. It looks like one of these should work.
Okay, noted that, and added all the silkscreen annotations. Hopefully I haven't missed any. Here is the result:
4. USB-C Charge Input Module
So the next cab off the rank for this one, is the USB-C charge input module. I'm planning to use the cheap-off-the-shelf USB-C Power Delivery Boards available everywhere online, e.g.:
The exact layout of these modules seems to change with the day of the week, and the ones I have here look similar but not identical to any of those. This includes the exact positioning of the power output pads, which is really annoying.
The ones I have here are 11x17mm, with 1mm extra over-hang for the USB-C connector. But I've seen others as large as 13x27, so I should probably try to make the module allow for this variety in size, even if the actual module I design takes only the exact model I found. As there are no other parts on the module, it won't be hard for people to redesign.
Importantly, they aren't even standardised on whether the + and - terminals are on the left and right. So apart from making the module big enough, there's no real point in me trying to make it accept the different flavours of these things. But even so, I'll do something fairly simple, in that I'll make the pads for it long and wide, so that some random selection of these things will work on it. They all have hole-through for the power out, which can be fairly readily tacked down onto tinned pads on the module, so the pads on the module don't need to be precisely placed. I'll go over this more when I get it fabricated and assemble it using the modules I have here.
So apart from the topological angst, this module is super simple, as it only needs two pins: Power out and GND. There really is nothing more to it, and this is the result:
On the PCB layout the huge pads I have created are quite obvious. The white guide lines on the silkscreen are there to help get the module correctly centred and straight, so that if the USB-C port has to fit through a hole in a case, it can be lined up easily.
This actually brings up an important point for these castellated
modules, in that the ones with connectors on the edge of the PCB have to
be positioned accurately on their carrier. In practice this probably is
easiest done by inserting the edge connector through the case with the
carrier board under it, and then solder tacking one or two pins down, or
failing that, taping them together, so that they stay in relative
position during soldering. If doing it that way, I'd strongly recommend
soldering just one pin, and then making sure it's still correctly
positioned while it's easy to deal with.
5. Power Source Isolation Module
I thought I had written up some notes somewhere on how to deal with this elegantly. But apparently not. Anyway, the problem boils down to accepting power via USB-C, the solar/dirty power input (6V -- 30V) or the internal solar panel as fall-back.
After having a chat with ChatGPT about it, it looks like something like the https://www.analog.com/media/en/technical-documentation/data-sheets/ltc4417.pdf might be a good option -- it allows selection between three power sources, with separate under and over voltage thresholds for each, including prioritisation. It accepts inputs from 2.5V to 36V, so that all sounds good so far. Let's look at a typical application for this IC:
So basically the IC senses the input voltages, and then picks which external set of MOSFETs to turn on. It also provides output signals that let something else check which voltages are currently valid, and thus allow it to infer which source is being used. This will be handy for indicating charge status, and showing the user which source is being used for charging, and which other source(s) could be used for charging if required.
The IRF7324 MOSFETs that they recommend are a problem, however, as their breakdown voltage is only 20V. We need >30V, ideally >40V or higher, just to make sure.
The datasheet recommends the Si4909DY as an alternative when higher maximum voltage is required. So we should probably use that.
We do have another problem, though, in that if the input voltage spikes, e.g., from automotive power sources, then we can still fry things. A Transient Voltage Suppression (TVS) Diode can help us with short spikes, by routing them straight to GND. For longer duration spikes (and also for the shorter ones) we might want to have an N-channel MOSFET that we activate when we are over-voltage via the Zener.
For example, to keep the MPPT charger comfortably below 28V, we could use a BZX55C27 with a 10K resistor to provide 0.1mA of current to switch an N-channel MOSFET to clamp the power rail to GND, which would then cause excess current to flow, while also reducing the output voltage until something like a PPTC self-resetable thermal fuse kicks in and reduces the voltage and current that passes through it.
We should only need all that on the dirty DC / external solar input, as USB-C is well-behaved, and the internal solar panel has an open-circuit voltage <10V.
So let's start piecing the schematic together for this. I've laid out the schematic with the MOSFETs and the selector IC. But I need to work out the resistor values for the selector logic. I also need to make sure that the priority order is right. I can safely use the same resistors for all three, allowing 5V to 28V. That said, increasing the minimum voltage for the dirty DC a bit might make sense, if it is using a large panel, and gets pulled down to 6V or so, it might not be very efficient. But I think I'm overthinking things. Also, there is great advantage in having common resistor values to ease assembly etc.
The under and over voltage comparators compare to 1V, so we need a resistor ladder that results in V_UV = 1V @ 5V, and V_OV = 1V @ 28V. I ended up messing about with a little spreadsheet to find values that are close enough. It looks like R3 : R2 : R1 = 246 : 51 : 11 gets us V_UV = 1V @ 4.97V, and V_OV = 1V @ 28V exactly. So let's scale those up. 246K, 51K and 11K are all common values, so we'll simply use those.
So this is what this part looks like:
So that's that part. Let's now turn our attention back to the input spike protection on the dirty DC. We can use a Zener diode to detect when the input voltage goes >28V. If we then put a resistor below the Zener, we can feed its output into the gate on a Thyristor, which is a bit like a sticky MOSFET transistor. It we put the Thyristor between the input voltage and GND, it will crow-bar the input rail to GND if the input voltage climbs above 28V. A pull-down resistor on the Thyristor's gate will help prevent it triggering erroneously. If we then put a PPTC resetable fuse in place, when the Thyristor triggers it will activate, and everything should come to a grinding halt, instead of letting the smoke out.
Okay, except that this idea has a problem: The resetable fuses have a really slow activation time -- like ~30 seconds. So instead let's look at having an N-Channel MOSFET that allows the power in normally, then use the Zener diode to develop a gate voltage to switch the MOSFET to isolate the power from the thing. Then we just need to have a Zener and MOSFET that have the maximum voltage tolerance possible.
Annoyingly there is no model for the IFP067N20NM6ATMA1 in either digikey or snapeda. Did I mention that I hate having to make footprints and symbols myself? Fortunately mouser.com seems to have one.
Okay, so for the TVS/Zener diode, it looks like a SMM4F24A-TR should work. It has a threshold voltage around 24V, add the 3.5V gate voltage for the MOSFET, and it should start switching the MOSFET on to disconnect the power source at around 27.5V.
Well -- after all that, I've had to redesign it somewhat, when I realised that the MOSFET needs more sophisticated control. I decided to try out Grok 3 Beta in the process, and was quite impressed at its ability to look at just a photo of my schematic and critique it. It wasn't perfect, but it was a considerable help at designing the MOSFET-based over-voltage protection circuit.
Here's how the whole thing looks now:
The top-half is the prioritised input selector, while the bottom left part is the over-voltage protection for the dirty DC input, so that it can handle "load dump" conditions etc on automotive power, for example, that can spike up to 60V or so. I've designed it so that it should tolerate 200V. Basically if the voltage gets near 28V, it will disconnect the negative terminal using an N-channel MOSFET.
After that I realised that I also need to add reverse-polarity protection -- which will basically work as an inverted version of the over-voltage protection, but with a P-channel MOSFET. But this is tricky, because P-channel MOSFETs have a higher resistance when conducting, which means more heat generation. The SQJ457EP-T1_GE3 looks possible, and would generate ~0.5W of heat when 5A flows through it, and costs less than $2.
So now it's time to lay out the PCB. Getting the components laid out is the first part, then we need to know which signals we need to expose on pins, which in this case is:
So that's 11 pins. I'll make a module with a few extra pins, just to make sure, since it's a little bit of mucking about if I discover I need extras after -- but not too much, since I made the procedural module footprint generator. Then we just need to know the size of the cut-out area.
Except that as I went to look at the size, I realised that I was missing a few 3D models for components, and when I added in a couple of those, I found they were either wrong, or for parts that weren't actually available. That's the peril of using AI to help design it :/ It'll be fine, but it does mean I won't get it done tonight.
Well, it's been quite the slog, but I now have the PCB layout finalised:
6. Solar and External Power Connection Module
Whew! I'm glad that the remaining module for this set of requirements is
super simple -- just a connector for the battery (but I'll probably
throw on the connectors for the internal solar panel, as well as the
external dirty DC / solar power. I'm currently undecided on the best connector for that. Maybe I'll just make it a bog-standard DC barrel jack, the same as on the MEGA65, so that you can charge the MEGAphone using a MEGA65 power supply as well. But part of me would like to have something a bit easier to adapt a solar panel onto. But probably all the good options are a bit physically big, even for the MEGAphone.
For the DC barrel connector, I'll go for something compatible with the MEGA65 power supply, just for commonality and convenience. While I'm at it, I might have a go again at finding the connector for my Librem 13 laptop, whose connector got damaged on a flight when someone tripped over the cable. I emailed Purism to ask them the exact part number, since I'm sure it's an off the shelf component, but didn't ever get a response from them. I just opened the laptop up again now to check the part for markings -- but it has none. I did in the process, however, discover that it's the power supply plug rather than the jack in the laptop that is faulty -- so that's a much easier repair.
(I also had a look at one of the USB-C PD modules I'm planning to use for the MEGAphone, and it looks like there is space enough there, if I file out the connector cutout on the case, and file down a positioning lug for the old barrel jack, and make some other retainer for it. Tempting! But that's a squirrel that I'll be ignoring for now.)
As expected, this board was fairly quick and easy to do, because of it's simple nature:
7. Internal Battery Connector
This is an even easier one. We just need a 2-pin screw connector or similar to anchor the unterminated leads from the internal battery, which could be fairly fat copper. The main nuisance is that such screw terminals are quite tall, but we should be able to accomodate them somewhere, even if we need a cut-away on the stacked PCB above it.
There really isn't much to say about this. The board is just a convenient way to mount a connector.
8. Requirements Verification
So
that's the last of the boards that we need to satisfy these
requirements -- but let's make sure that we have everything covered off:
Requirement 1.1.2.3: Proof-of-concept charger module based on a given USB-C power delivery module (no need for data). -- Satisfied in 4. USB-C Charge Input Module Requirement 1.1.2.4:
Power isolation mechanism to allow conflict-free operation of the USB-C
and alternative power inputs to the MEGAphone (back-flow and
over-voltage protection, for example) - Satisfied in 5. Power Source Isolation Module Requirement 1.1.1: Integrated LiFePO4 or other similarly safe and robust chemistry battery cell. -- Satisfied by 7. Internal Battery Connector Requirement 1.1.2.1: Integrated battery capacity of at least 30Wh. -- Satisfied by 7. Internal Battery Connector (as it can take a massive battery) Requirement 1.2.2.1:
Means of accepting (including connector) dirty DC power between ~9V and
~40V28V using a MPPT to maximise efficiency with a solar panel. -- Satisfied by6. Solar and External Power Connection Module and 3. Solar MPPT / Dirty DC Input & Battery Charger Module Requirement 1.2.3.1: Means of accepting DC power from integrated solar panel at input voltages of 2V6V to 20V with a MPPT.-- Satisfied by6. Solar and External Power Connection Module and 3. Solar MPPT / Dirty DC Input & Battery Charger Module
In short, we have covered all of the requirements with this set of five modules -- even though they are all grouped as a single module in the milestone. Oh well. At least they are done. The MPPT and power input selector and over-voltage / reverse polarity protection circuits are actually the two modules that scared me the most in this whole project, as my expertise is really in digital systems, not this analog power-electronics world. But I guess I can't really relax until I've built them and confirmed that they work properly.
The schematics are all in the modules/ directory in the source code repository, as usual.
