The HDMI-compatible audio drama might finally be coming to an end.
Adam has kindly written an implementation that uses direct FPGA lines instead of the ADV7511 chip, which we have confirmed works on the QMTECH Wukong FPGA board.
The Wukong board is quite amazing: It is a complete 100T based FPGA board, that you can buy for less than the price of a single 100T FPGA! And it has a direct HDMI output interface, which is good for our debugging purposes.
This makes it a much better option than the Numato Mimas A7 board we used previously for this, as that board had only a smaller FPGA that couldn't fit the whole MEGA65 design, so I was limited to testing the HDMI video output. Being able to fit the whole MEGA65 design means that I am able to get a head-start on the changes to the VHDL that I need to make for the MEGA65 R3 PCB, that will also have such a direct HDMI interface.
The starting point was to make a target for the Wukong board in our build environment. Making new targets really just consists of making three files:
vivado/TARGET_gen.tcl
src/vhdl/TARGET.vhdl
src/vhdl/TARGET.xdc
The first is used to generate the Vivado project files, and is rather boring, being mostly a list of files. I even started some work on auto-generating those, but for now, it is just hand done.
The second is the top-level VHDL, where we connect all the interfaces of the MEGA65 design to the external hardware pins, and also add any optional pieces that are required, e.g., HyperRAM controllers for boards that have HyperRAM.
The third is the list of FPGA pins and their names.
That's the easy part. After a day or so of fiddling, I had a bitstream to try.
The next fun part is that on the current revision of the Wukong board, there is no integrated JTAG-over-USB. This is annoying, as it means that you have to connect an external JTAG programmer to the header. I used the TET0004-02 Arrow programmer that I normally use for programming the keyboard CPLD in the MEGA65, and workout the pin arrangement to connect it up. Also, the Wukong board has no microSD slot, so I used a PMOD microSD adapter I had laying around. Then add the USB mini port for the serial-over-USB for the MEGA65's serial monitor interface, and then a 5V power supply, and I had it all going.
That's right, including the HMDI out, we need four different connections to the board to make it work. They could have easily done the same as the N4DDR board or the TE-0725, and allowed it to be powered over USB, and to do JTAG over USB, so that only USB and HDMI would be required. I've contacted QMTECH about this, and they gave me a nice polite response indicating that they would likely implement at least some of that on their next revision.
Anyway, here is what the spaghetti-monster cable-salad looks like with the current revision, with close-ups of the connectors that are important to get the correct pinout for:
It took a little fiddling to get the pin-out for the microSD right, but that now works nicely. We also get a good stable HDMI image.
What is currently the problem, is that I can't switch it to C64 mode, as the CPU goes off somewhere funny when tries.
Tracing through, it goes into the DOS ROM to do the normal C65 internal drive DOS setup up. It's interesting going through that code. It is quite inefficient in places, such as how it initialises buffer memory using tight loops, rather than using the DMAgic controller, for example. The code to extract drive error messages is also quite fascinating in just how long it takes to get such a message out. Neither is really a problem, except when you are single stepping through the startup trying to find a problem...
Okay, after nearly giving up, I have traced out the otherside of the DOS setup, and got to $FCF5, which is the C64 ZP and low memory setup routine. While stepping through that, I thought I would just check if the $01 CPU register was set properly to map KERNAL and BASIC. It looks like it is, but there seems to be something funny going on: The BASIC ROM is not visible at $A000. Rather, I see a slab of $00's. But it isn't the RAM at $A000 showing through, because if I modify the RAM, it doesn't show up in the ROM area. Nor is it from $1A000 or $3A000. I did make sure that the BASIC ROM data is loaded at $2A000.
Ah, but it could be that it thinks that there is a cartridge... That would indeed be the case, it seems. I had tied the /GAME and /EXROM lines to Z for high-impedance, but in the absence of an actual driving signal, that results in a low rather than high signal when read.
So I made some changes to the VHDL to fix that, but in the meantime, I thought I would try to use the Hypervisor option to disable cartridges to work around it temporarily and let me get to C64 mode where I could test audio, without having to wait for the new bitstream. That was all going fine, until I started smelling magic smoke. I still don't know that happened, but I think the JTAG programmer must have rested on something in a way that shorted it out. Anyway, as you can see, the power lead got a bit melty:
Fortunately it didn't let all of the magic smoke out. Its a bit tricky to get it to programme the FPGA now, but it still usually works.
So I was finally able to switch to C64 mode, and load a game that I know makes noise at the intro screen, and confirm that, yes, we now have audio coming out of the HDMI TV I have connected up to it:
The audio sounds nice and clean, if a little quiet. But we can easily increase the gain in the MEGA65's audio mixer to compensate.
It's hard to test much more on the Wukong board right now, because I don't have keyboard or joystick inputs hooked up to it. I do have a spare MEGA65 prototype keyboard here, but not a cable to hook it up. And, besides, I don't really need to for testing purposes.
What I will do, though, is make a D81 to be the default disk image, and put an auto-booting version of Commando or maybe our MOD file music player, so that its easier to test other screens. Then its just a case of flashing the bitstream to the onboard flash memory, and we will have the board all setup for easy testing of different TVs and monitors.
Probably using the TE0790 JTAG programmer would allow me to flash it from vivado, but after letting magic smoke out of the Arrow one, I'm not that keen on risking the TE0790, since ordering replacement parts in the middle of the Outback takes weeks. I'll have a think about how I work around that soon.
But the main thing is that we now have an updated HDMI audio test platform, that I can use to test all the TVs around the place up here -- in particular the Sony one, as those are renowned for being picky about the audio framing.
This post is about something that I have had in mind for a long time: A couple of years ago (or possibly more), someone pointed out to me that the DMAgic controller could in theory be used to paint textures, since texture painting really is just a matter of copying a texture onto the screen. Well, and scaling it appropriately. And dealing with the screen memory layout, so that each successive write goes to the next pixel. Luckily, its possible to do both using some quite simple tricks.
First, let's look at the pixel writing first. First up, remember that the MEGA65's VIC-IV has no frame buffer. Rather, just like the C64's VIC-II, the screen is usually made up of 8x8 character cells. The main difference between bitmap mode and text mode is whether you can choose which cell appears where, or whether they appear in a fixed arrangement. This is actually a strength of the VIC family of video controllers for a Wolfenstein type engine, where we need to draw vertical stripes of pixels. Let me explain:
First, lets consider a toy screen with only 4 columns and 4 rows:
ABCD
EFGH
IJKL
MNOP
This is the normal arrangement in bitmap mode, with ascending characters going across the screen. So to find the address of the next pixel down in a column we have to work out whether we are crossing a character boundary, and if so, add enough to the address to skip the rest of the characters on the row. But if we use text mode, and put our pixels in the character data, we can arrange the screen differently, striping down the rows instead of across the columns, like this:
AEIM
BFJN
CGKO
DHLP
Now, if we want to advance one pixel down the screen, we always add the same amount, i.e., the number of bytes to skip down one row in a character. On the VIC-II, that is always one byte, as one bit defines each pixel. However, for a nice looking engine, we would like to have more pixel colours. So for this engine, I am using the VIC-IV's "full-colour text mode". You can read more about it in the MEGA65 Programmer's Reference Guide, but the bottom line is that each pixel of a character is now represented by a byte instead of a bit. This means a row of character data consisting of 8 pixels now takes 8 bytes instead of 1 byte (8 bits). So to go from one pixel to the next, we need to tell the DMA controller to add 8 to the address every time when it is generating the address to write to.
That's the writing side. So lets think about the other half: Reading from the texture, and scaling it. The first observation is that the scaling should be done on the reading side, as we always want to write to each destination pixel exactly once. Thus the solution is to change how the source address is calculated for each new pixel.
If the texture is being drawn at 1:1 scale, then its simple: We can just read each successive byte of the texture data, and write it out with our x8 address generation when writing to the screen.
If the texture needs to appear bigger on the screen than it is in memory, then we need to increment the source address only sometimes, so that the same pixel values get repeated.
Conversely, to draw the texture smaller on the screen than it is in memory, then we need to increment the source address by more than 1 on average, so that we skip some pixels. (Ideally we would calculate some average of the pixel values, but that's too much work for our poor 8-bit machine, and certainly too much to ask the DMA controller to do for us).
Our solution to this problem is to allow the DMA controller to increment either source and/or destination address by between 1/256th of a byte and 255 bytes each time. This way we can skip 8 bytes at a time on the writing side, and quite smoothly scale the texture we are reading on the reading side. There will be a very small fraction of a pixel error at the end sometimes, but this will be a consistent error, and thus won't be noticeable on screen.
What is best about this, is that we can scale and draw a texture of any resolution at any zoom on any height screen, at a constant time cost of 2 cycles per pixel drawn. For a 320x200 screen, that means about 128,000 cycles. Given that the MEGA65 has at least 675,000 cycles per frame in NTSC (or 810,000 in PAL), this means that we could in theory draw the screen at full frame-rate, with some margin to spare.
This compares very favourably with any other possible method, which presumably would have to involve at least one load and store operation (and probably a lot more), resulting in a cost well north of 9 cycles per pixel drawn. I saw the difference first-hand when I was porting the engine, as I didn't initially have the DMA painting implemented. Without the help of DMA, it would take several seconds to draw a single frame, i.e, about 200 PAL frames to draw the screen. With the DMA painting implemented, I was able to achieve a frame rate of up to 10 frames per second, i.e., about 5 PAL frames to draw each frame. This means that our DMA method is about 40x faster than without! That's on top of the 40x speed up thanks to the CPU speed.
So for this particular case, we are able to outperform the C64 by a factor of about 1,600x! That's the power of custom-chips. Interestingly we are also a lot faster than the best Amiga Wolfenstein-type engines I have found. This is to be expected because the full-colour text mode effectively gives us a "chunky" video mode -- but with the extra advantage that we can arrange the screen the way we did, so that the striding between addresses is constant, and thus faster than even a PC of the era could achieve.
But we are getting ahead of ourselves here -- we started talking about how to do the texture painting, and are now talking about a complete engine, without me showing you the steps in between...
So let's talk about Wolfenstein type-engines and how they work. They were the first successful 1st person "3d" engines that allowed the user to look in any direction. They did this by "casting" light "rays" in various directions, and working out where they hit, and then drawing that thing. They also had to work out how far away the thing the light ray hit is, so that they could work out how big to draw it, to simulate the impression of distance. One light ray was cast per column of pixels on the screen, thus the interest in drawing those columns quickly.
The algorithms and optimisations that they make to draw quickly are quite interesting, and there is plenty of information on them out there, as well as example engines. I tried a couple before settling on one that was designed for fast fixed-point calculation (as compared to floating point), since the MEGA65 doesn't have a floating point math unit. I then forked it on github, and set about making work on the MEGA65.
Looking back at the commit log, it took less than 4 hours to adapt it to draw a display on the MEGA65, including using the DMA drawing method described above. It took a further couple of days to fix a few bits and pieces, implement movement around the map using the cursor keys, including bumping into walls and the like.
The sky texture which adds a great feeling of realism and helps to orient the player in terms of which direction they are facing turned out to be super easy to do: Because the sky is "vertical" we can just have a loop texture, and draw it in each column of the screen, down to where the textured wall should appear. The floor on the other hand is a pain, because in this view, the ground is horizontal, and so drawing it in the same way as the sky looks quire weird when you walk forwards or backwards. When you stay still it looks pretty good, though. To help make it look better, I rotate it, but its still not perfect, and there are bugs in the way I am doing it on top of that, where it scrolls in the opposite direction it should when you walk in certain directions. But its a start.
The keyboard input was quite interesting to get right, as for intuitive movement, I wanted to know whether particular keys were being held down, not just pressed. This meant that the MEGA65's hardware accelerated keyboard scanner was only of limited use, because it doesn't indicate when keys are released. However it does provide separate access to the keyboard matrix in a convenient way, similar to how the C64 KERNAL scans the keyboard, but in a separate register pair, so that you can scan joysticks and keyboard simultaneously. As the MEGA65 keyboard is fully diode protected, it is possible to test the state of every single key independently. Thus I implemented both WASD and cursor key control. For good measure I also implemented joystick in port 2. I also implemented 1351 or amiga mouse in port 1 to control the direction you point.
As the sample map that the engine I used was a bit boring, I decided to replace it with a random maze generator. I spent more time on that than intended, until I eventually found a good and complete pseudo-code description that allowed me to implement a working maze generator. The result is in mazegen.c in the repo.
As I wanted to support mazes of various sizes, I decided to use a slab of memory above 64K to hold the information about each maze cell, and make a routine to look up values from that using the lpeek() function in the MEGA65 libc. However, I found that the drawing speed of the engine went through the floor from about 10 FPS to about 2 or 3. This is because checking if a particular block is empty or not occurs right in the heart of the ray casting algorithm. Thus doing a complicated memory fetch was slowing it down.
Worse, as CC65 is horribly slow at passing arguments to functions, that fetch was taking even longer. So I implemented a little bitmap that had just whether a given cell was empty or not in it, and put that at $C000, so that a simple memory read could be used. I wrapped that in a C Pre-processor macro so that there would be no function calls, and the frame-rate improved quite a bit, up to maybe 6 or 7 frames per second.
Along the way I also changed a few routines that were performing multiplications to use the MEGA65's hardware multiplier. But the core routines are still in fairly boring C. I suspect that with some hand-tweaking, it should be possible to make it quite a bit faster than it is now.
Once I had the core engine working, I started thinking about how I would do titles and other text that needs to appear in the game. I could draw them over the rendered 3D display, but that would cause a lot of problems. At the moment, because the rendering is so fast, it looks fine to just render from left-to-right continuously, without using double-buffering. This is great, because I don't have a easily have a spare 64KB of RAM to dedicate to the double buffer (I could ditch the ROM to gain 128KB, but that's a pretty desperate measure). But its not so good if we want to draw text over the top, as we would have to do a second layer of rendering, which would slow things down, and risk making a glitchy display.
Instead I used one of the VIC-IV features which is the ability to draw on a raster more than once. That is, we can basically ask the VIC-IV to do in hardware that which would be a pain to do in software. This works because the VIC-IV has a single raster line buffer into which it renders. If there is enough raster time, its possible to tell the VIC-IV to go back over the raster, and render some more stuff. If you set the appropriate flag, any background pixels in the material being drawn over the top are not drawn, effectively keying the new foreground content over the existing material from the first pass of rendering.
Don't worry if that sounds horribly complicated. Just think about it as a kind of "text sprite": We can put an extra layer (or more!) of text over the top of whatever is being displayed, and have the VIC-IV composite it for us in hardware.
This is the feature that I used to make it possible to put text-based titling over the 3D display. This is used in the intro screen, as can be seen in the video I made of the core of the game that I already have working:
This works by telling the VIC-IV that each line of the screen has 80 characters, the first 40 of which are the ones arranged in the vertical stripes for efficient texture painting. The 41st is the "GOTO" token that tells the VIC-IV to go back to near the left edge, and start drawing the 42nd to 80th over the top of the first forty. In reality there are 2 bytes per character, for a total of 160 bytes per line, beause we need to use 16-bit text mode to access this feature (and to have more than 256 characters on the screen, which is needed for the 3D display which uses 1000 different "characters" to make our optimised bitmap layout).
Thus we need to tell the VIC-IV that each line has 160 chars. We do this by writing 160 into $D058, and 0 into $D059 (because there are less than 160 chars per line. We then put 80 into $D05E to tell the VIC-IV to draw 80 characters on each line. The ability to set these two values independently is necessary for 16-bit text mode. But it is also powerful for making screens bigger than the display area, that can then be hardware smooth-scrolled. This would be very useful for making a side-scroller, for example, as the entire map could be setup as a single giant screen, of which only part is visible at a time.
For the curious, the routines with "overlaytext" in their names in main.c have the magic. Basically it is a lot of address calculation to work out where to put a character to make it appear in the right place. Otherwise, there is the nice routine overlaytext_line_x_position(). This one is interesting, because it makes it simple to set the horizontal position where the compositing of the overlay onto the rasterline will occur. That is, it can be used as a kind of hardware smooth-scroll on a line-by-line basis. This is what is used to slide the title screen text off the right hand side when you start the game.
As well as general VIC-IV and DMA hacking, we needed random numbers for the game. So I wrote routines for generating random numbers. This includes routines that try to gather thermal entropy from the FPGA to produce true random numbers (although we still have to test that this is reliable). That process is slow, and sometimes you want reproduceable sequences of numbers, say, for consistently producing the same random maze. So I also created srand() for seeding a pseudo random number generator, and rand8(), rand16() and rand32() for obtaining 1, 2 or 4 byte random numbers. Each takes an optional argument that can be used to obtain numbers in a limited range. It works by multiplying the random number by the supplied argument using the hardware multiplier, and returning the upper bytes of the result. The result is nice and fast.
You might have noticed that in the video the game was started from C64 mode. Another little piece of work we have done is to create a wrapper programme that can be used to make any C64-mode oriented programme launchable directly from C65 mode or C64 mode. This is interesting in its own right, so I'll blog about it separately when I get a moment. But suffice to say that the current version of MEGA MAZE can be started from either C64 or C65 mode using the same programme. By naming this AUTOBOOT.C65 on a D81, we can even auto-boot the MEGA65 to run it, if it is on a disk in the floppy drive (or on the D81 image that is mounted by default).
Finally, to get music playing in the game was a bit interesting, because there isn't really space left in the first 64KB of RAM for a SID file. There is fortunately space for an interrupt handler. So I wrote a tiny little interrupt handler that banks in some of the 2nd 64KB of RAM that contains the SID file, calls the play routine, and then puts everything back before resulting the interrupt handler. It looks like this:
I put it in a C array like this, because I copy it into place in $0340, rather than trying to have an interrupt handler call a C function, which is known to be messy in CC65. The routine itself is very simple: First, set the bottom 8KB slab to be banked, and the bank address to be +$1C000. This causes a SID file stored at $1D000 to be mapped at $1000 (since $1000 + $1C000 = $1D000). It then calls the play routine for the SID file, and then clears the mapping, acknowledges the raster interrupt, and resumes the normal interrupt handler.
There is a trick, though: This little routine has to also be installed at $1C340, so that when the first MAP instruction executes, the rest of the routine is visible in the new memory mapping. The routines that handle all this are in music.c of the raycaster repo. Those routines also handle parsing the play routine address out of the SID file, and patching the above handler to use the correct address:
lcopy(music_irq,0x340,63); lcopy(music_irq,0x1c340,63); // Set IRQ handler to music routine POKE(0x0314,0x40); POKE(0x0315,0x03);
// Disable CIA interrupt, and enable raster interrupt POKE(0xDC0D,0x7f); POKE(0xD012,0xFF); POKE(0xD011,0x1B); POKE(0xD01A,0x81); POKE(0xD019,0x81);
asm ("cli"); }
I realise that this has all been a bit of a whirl-wind tour, rather than deeply worked examples of the various features. But hopefully it is enough to get you thinking about some of what is possible on the MEGA65, and some example code you can look at, as well as pointers to where it is documented in the https://github.com/mega65/mega65-user-guide repository.
Just a short post this time, but a very exciting one: The first commercial software title has been announced for the MEGA65! Hibernated 2 will released on 3.5" disk for the MEGA65:
Both the author and Stefan Vogt and PuddleSoft are enthusiastic supporters of the MEGA65, and thus it our great pleasure to be able to announce them as being the first retro-game house adding support for the MEGA65. Here's a screenshot from when I was helping to get it running:
What is particularly exciting about this particular release, is that it uses a Z-Machine text adventure engine, Ozmoo, that I ported for the MEGA65 last week. This means that it is possible to make MEGA65 versions of a large back-library of text adventure games, all enjoying 80 columns and 40MHz CPU speed. While the 80 columns makes an obvious contribution, the CPU speed is actually super-helpful here, as the Z-Machine is notorious for running slowly on 6502-based systems. At 40MHz, though, it feels super responsive.
Just to show how easy it is to use our port of Ozmoo to create versions of existing text adventures, here is a screenshot of The Hitch-Hiker's Guide to the Galaxy:
We are hoping that this will be the first of many official software releases for the MEGA65!
Just a quick post to report on progress on the integrated sprite editor that we are building as a plug-in to the Freeze Menu. The reason for doing this is that unlike BASIC 7 on the C128, the C65's BASIC 10 doesn't include a sprite editor.
We have already made this plug-in framework, where the Freeze Menu can call helper programmes for the audio mixer. So it was really just a case of making the sprite editor itself.
And this is where one of our volunteers Hernán comes in: He has been working on the actual editor. Together we merged that in as a plug-in to the Freeze menu, and hooked up all the plumbing, so that exiting the sprite editor returns to the freeze menu. We also jointly implemented simultaneous joystick and mouse support, so that its is super comfortable to use. To enter the sprite editor, you simply press S from the familiar Freeze Menu. You then get something like this:
You can then use the keyboard cursor keys to move around, and toggle pixels. You can also see that there is provision for multi-colour sprite modes. Now, the screenshot tool doesn't currently show sprites, which I only just realised while pasting the screen-shots in. This is annoying, because it means you can't see the sprite pointer in the following, where I have used the mouse to move the cursor and draw:
One of the nice things that has come from this, is that we have made library functions for using the mouse as part of the mega65-libc. Hernán also wrote simple console output library functions for drawing the display. This means that we have a new set of APIs that are easy for other programmer's to use in their own programmes. When I get the chance, I'll start documenting the library functions to be included in the MEGA65 Book, and likely in a smaller separate MEGA65 Cross Developer's Reference Guide.
Anyway, the editor is now at the point where the hard parts are mostly done. Next steps will be to make it actually edit the sprites from the frozen programme's memory, rather than from in the memory of the sprite editor process.
In short, NLnet have kindly agreed to fund a body of work on advancing the MEGAphone, and by implication, advancing the MEGA65 project. This means that I will be spending a considerable amount of time in several pieces between now and early next year working on the MEGAphone. This means I am going to be spending more time working on the MEGA65 as a whole than I otherwise would have, so it's a positive all round. But I understand that some folks are only interested in the MEGA65 as a retro-computing platform. In which case, feel free to pay less attention to these posts. That said, they will still be very much focused on fun retro development, and solving very much real world problems with such a system. After all, its not every day that someone makes an 8-bit smart-phone that can communicate via ultra-sound!
If borders open, and international travel becomes feasible again, this will be done in Darmstadt with the rest of the MEGA65 team, but until then, I'll continue to work from here, since there isn't really any alternative.
Speaking of the effects of COVID19, this has also slightly tweaked the work plan, as NLnet are keen for us to look at how the idea of a sovereign and fully open phone can be used to produce a more privacy-protecting and less error-prone form of contact tracing. In particular, they are interested to know how practical implementing near ultra-sound communications would be, to see if it makes sense as an alternative to bluetooth-based proximity detection. Ultrasound has some nice theoretical advantages, like not working through walls or other barriers that are likely to also be effective barriers against virus transmission. Thus it has the potential to reduce the false-positive rate.
Exploring ultra-sonic communications is something that I have been wanting to do for a while, and was already on the plans fora the MEGAphone as a means of resilient communications. For those who remember when I first talked about the MEGAphone, they might recall that it already has a bunch of weird communications modes available, including an IR LED that can probably turn TVs off from 200m away. It also has not just one microphone, but an array of 4.
The microphone array was intended for making it easier to to cancel background noise, and detect the direction a sound is coming from, so that speaker-phone mode could do various party tricks with it. However, having read about ultrasonic communications around that time, I chose microphones that are sensitive well into the ultrasound range. So, its quite possible that we may be able to achieve ultrasonic communications with the MEGAphone. And that is the first milestone in the NLnet project:
1.
Assess near ultrasound capability of existing MEGAphone prototypes.
There already exists
several revision 1 hardware prototypes of the MEGAphone, that form
the basis for forward activity on the project. These include MEMS
microphones and amplified speaker output functionalities that have
the potential for near ultrasound communications. The purpose of this
sub-task is to assess the feasibility of this, and gain an
understanding of what may be possible. It is entirely possible that
this will prove infeasible with the current hardware, in which case
the reasons and potential remedies for this will be considered for
inclusion in a future revision of the MEGAphone hardware.
Milestone(s)
Examine the
existing components of the MEGAphone r1 and r1b prototype hardware,
in particular the MEMS microphones and speakers, to determine their
theoretical suitability for near ultra-sound communications.
Determine the
ultrasonic frequencies and bandwidth that are likely to be possible
to use, and consider the constraints that this is likely to place on
any protocol designed to use this facility.
So my first goal here is to look at the speakers and microphones in the MEGA65, and, and see what their theoretical properties are, and whether they have the potential to be used for ultrasonic communications, and if so, what frequency bands we expect to be able to use. Of course, we could also consider dedicated ultra-sound communications, but the goal here is to use existing speakers and microphones, rather than increasing the bill of materials of a phone. So let's get started:
`
Microphones
The MEGAphone R1 prototypes have four MEMS microphones. As physically very small structures, they have a natural resonance that is well into the ultrasonic frequencies. The ones that we are using are the SPW0690LM4H. According to Table 2 of that datasheet, they have a Resonant Frequency Peak at 26KHz. Also, Table 2 tells us that at 15KHz they are 3dB more sensitive than at 1KHz. Thus we can expect that they will likely be competent up to probably around 40KHz or so. Figure 9 provides some more information, showing the frequency response:
Here we see a peak at 26kHz as promised. We also see that sensitivity is quite reasonable all the way to the 80kHz limit they show, and in fact from ~72kHz the sensitive is above the sensitivity of the acoustic range. So while somewhere around 26kHz would be ideal, with a benefit of close to +20dB versus the acoustic band, any ultrasonic frequency up to at least 80kHz should be usable. So the receive side is not likely to be a problem.
Speaker
The speaker in the MEGAphone is a CMS-40504N-L152. As this is not on the PCB, we can in principle easily swap it out for another. However, hopefully we won't need to do that, because these speakers are simply fantastic for the MEGAphone. They are super-loud and take up to 2W for loud ringing and playing games, and have good frequency response thanks to their relatively large 40mm diameter. And for all that, they are only ~5mm thick.
Here we have a less positive prospect: It claims a maximum frequency of 7kHz. This will presumably be the maximum frequency with reasonably flat response, above which it waill presumably roll off. Digging through the datasheet we find:
Well, this is much better than feared. Yes, above 7kHz, it is 10dB below the lower frequencies. But it isn't a flat roll-off. Rather there are a couple of interesting peaks, the first at ~9kHz, and then a second at what looks to be ~18kHz, after which there is a similar big drop-off. Unfortunately, they don't show any higher frequencies on the table. This is a bit unfortunately, as 18kHz is still audable for some younger people (I can still just hear 17.4kHz, and maybe a bit higher, and I am in my 40s).
But I am just about willing to bet that there is another peak at ~3x9kHz = ~27kHz, which would nicely coincide with our microphone's peak sensitivity. If possible, that would be a particularly effective combination. The question is whether the speaker is still loud enough, and whether we would start to get audible distortion. This will require some experimentation.
Either way, the frequency response around these frequencies is a bit sharp, so the bandwidth available is likely to be quite narrow -- perhaps less than 1kHz. This will affect what wave-forms we might try to use for a communications protocol. For example, frequency shift keying might be problematic, because allowing for variation of peak frequency resonance among the speakers and a bit of Doppler shift etc. Similarly chirped spread spectrum would be problematic, due to the very narrow frequency band available for the chirps.
Although for contact tracing, we can probably assume that if you are moving past someone fast enough to cause a large Doppler shift, you are probably unlikely to catch anything from them -- unless perhaps you are on a fast merry-go-round near a stationary person for a long time. Thus we can probably ignore the problem of large Doppler shifts due to high velocities.
Would other speakers be any better?
There are certainly other speakers that have higher rated frequencies, and thus might be capable of higher frequencies. We'll examine that a bit later, if it turns out to be necessary. This is because we need only a few metres range, and if we can achieve this with the existing speaker that is optimised for our other needs, then there is no point changing it. To determine that, we need to consider out total link budget, similar to if we were using radio.
Link-Budget Estimate
Lets now think about what the link budget at 18kHz and 27kHz would be. We will start by examining the microphone sensitivity and maximum speaker volume, and then subtract for the free-space path lost as the ultrasound disperses through the air.
According to the speaker data-sheet, we should be able to produce a signal of ~94dB SPL (sound pressure level).
The free-space path loss through air is relatively minor for the frequencies and distances that we are concerned about. According to this calculator, the path loss will be <1dB per metre. However, the total path lost due to the inverse square law etc, is more like 20dB over 1m, ~34dB over 5m and ~41dB at 10m
Finally, we have the sensitivity of the MEMS microphone to consider. Here I am having to stretch my understanding of the data-sheet -- so if anyone reading this spots any errors in my reasoning, please let me know, so that I can fix it.
The MEMS microphones indicate a signal-to-noise ratio of 82dB for near-ultrasound frequencies (around 20kHz) with a 94dB SPL (sound pressure level) sound source. We know that this should improve as we approach the resonance at 26kHz, so we will assume that we can detect down to 94 - 82 = 12dB.
Pulling this together, at a range of 10 metres, we should have a link budget of 94dB SPL - 41dB - 12dB = 41dB. If we allow 20 or 30 dB for multi-path interference and all the usual horrors of propagation, we should have a signal with an amplitude of 10 dB to 20 dB, i.e, 10x to 100x the background noise level. In reality, this might be somewhat worse due to interference from ambient noise sources, and muffling effects, such as having the device in your pocket or bag. That said, if we are aiming for 1.5m instead of 10m, there is better than 10dB margin to be gained. Also, all of this assumes an omni-directional sound source, which will not be true -- but it is a starting point.
However, overall, it seems like we should have a positive link budget at the end, with an SNR of somewhere around 10dB to 20dB. So for now, we don't need to immediately try a different speaker.
Constraints on Protocol Design
So it sounds like we should have a signal of a few hundred Hz bandwidth, and with a link budget of 10 -- 20 dB. Within this constraint, there is still considerable freedom for selecting a solution. Optimising that would require considerable effort and expertise beyond what we possess, or what is in this case actually required: This is because contract tracing requires only very modest data transfer rates. Each beacon can potentially be only a dozen or so bytes long, and need only be sent every minute or so. Given most guidelines are for 15 minutes of close proximity, this would allow significant redundancy in the beaconning, to help reduce the probability of false negatives.
If we allow for 15% channel efficiency for a simple ALOHA approach, and up to 100 people within range of each other at a time, this means that each beacon must consume less than 0.15% of each time step. For a 1 minute beaconing interval, this corresponds to an air-time of 90 milliseconds. Assuming we need a 32 baud synchronisatiaon preamble and a 64-bit random token, this would require a data rate of ~96 bits / 90 millisecond = ~1kbit / second. This seems totally achievable within the expected channel characteristics.
If the token consisted of a 48-bit unique token and 16-bit CRC, this would provide for robustness in the protocol, while still maintaining a very low false-positive rate. We can use the Birthday Paradox to compute the beacon collision rate with 2^48 tokens and not more than 2^32 simultaneous users of the system: The probability of any two users using the same beacon at the same time would be ~1/(2^(16/2)) = ~1/256. Thus we would expect of the order of 10^1 colliding beacons per day, globally. Assuming that less than 1% of users of the system test positive for the virus on any given day, this would result in ~10^(1-2) = ~1/10 false positive situations per day, i.e., of the order of one person per week being erroneously told that they had been in contact with someone with the virus. This is almost certainly below the noise floor of the viral testing procedures and various other factors. Thus we accept this false positive rate.
As can be read below, a channel bandwidth of around 1KHz seems quite possible -- provided that the other technical barriers can be resolved. Thus while careful protocol design and implementation would be necessary, the channel bandwidth would not impose any particularly troublesome constraints on the protocol design.
In fact, the very short range of communications that would be likely realised would result in rather more relaxed constraints than described above. This could allow for a reduction in data rate and/or longer packets containing more information, which could be used to effectively eliminate the remaining false-positive rate.
Conclusion and Recommendation
For the full detail on how the following conclusions were reached, read the "Appendix: Experimental Verification" section below, but the TL;DR version:
1. It is possible to perform bi-direction near-ultrasonic communications using the existing hardware of the MEGAphone prototypes.
2. There is likely sufficient bandwidth to implement a realistic contact tracing facility.
HOWEVER
3. My suspicions about the real-world practicality of this led me to go beyond the scope of the milestone and actually test the components, which identified several important factors, including:
4. The audible artefacts, limited communications range, and by implication, the power budget, of near-ultrasonic communications in this context is rather problematic.
5. It would seem that for all of its problems, Bluetooth is a better solution, due to its vastly superior performance in terms of range, lack of audible artefacts and existing wide-spread deployment.
THEREFORE
6. I do not recommend further pursuit of creating a near-ultrasonic contact tracing system using mobile phone hardware at this point in time. This conclusion does not impact on the creation of bespoke devices that avoid these problems through careful design.
IN ADDITION
7. While not well suited to a contact tracing use-case, we have established near ultra-sound as a quite feasible means of digital communications using off-the-shelf mobile phone parts. This may be of assistance to people operating in areas where electromagnetic communications are denied or heavily surveiled.
Appendix: Experimental Verification
It's all well and good to say that the above should work. But as we know, saying should in computer science usually means "won't despite all expectation". Or as my German friends like to remind me "eigentlich ist die stärkste Verneinung", i.e, "Actually" or "In Reality" is the strongest contradiction. Thus we want to make sure that we are not barking up the wrong tree, and want to at least observe and characterise the performance of the MEGAphone's actual near ultra-sound performance.
My approach here is to produce an ultra-sound tone using the speaker, and then pick that tone up using the microphone array. This will done on the MEGAphone using some software I have written for this purpose.
The first step is to make sure that the MEGAphone microphones really are sensitive to ultra-sound. For this, I used a tone generator app on my boring Android phone to make sure that I can pick it up. The MEGAphone R1 prototype has one dead MEMS microphone, so I had to fiddle to pick one that was working, and that was conveniently located where I could get to it. This let me test that I could receive a 20kHz tone. The signal to noise ratio (SNR) I didn't measure, because I don't know the actual SPL loudness of the phone's speaker that frequency, or what funny filtering the Android phone has. I did discover that my daughter can hear upto 19kHz quite fine, when she started to complain while I was testing ;)
So now the next step is to modify the MEGA65 test programme I wrote so that it can also produce the tone. This will be fairly easy, as the MEGA65's audio output is continuously integrated using a pulse density encoder (PDR), so sample rates up in the ultrasound range shouldn't be a problem.
Also, running the CPU at 40MHz gives us plenty of horse power to do this -- or at least I hope so. I will run a non-maskable interrupt at some multiple of the target frequency, driving a small sine curve table of values at maximum volume.
The interrupt loop will need to be quite tight, but it should be okay. Something like:
sampleplayernmi:
PHA
PHX
LDX $FD
LDA ($E0),X
INXaa
TXA
AND #$0F
STA $FD
PLX
PLA
RTI
That will take something like ~35 cycles, including the interrupt entry and exit, plus a few cycles jitter while the CPU finishes whatever instruction that the CPU was executing when the interrupt is triggered. So close to 40 cycles = 1uSec. Thus our over-sampled frequency can be up to 1MHz. If we have 16 entries in our sine table, then this allows up to 1MHz / 16 = ~64kHz as our maximum effective frequency. That should be sufficient, since we only need 1/2 that. Thus about 1/2 the CPU time will be spent on the interrupt, leaving the other half to read the microphone samples and visualise them.
Ideally we would synchronise the sample reading with the writing, as this would give us a display that is always synchronised with the tone we are transmitting. So I might rework the above to read and stash the microphone samples, as well. That will eat more CPU time, though, so I'll probably have to optimise it carefully. It might be, that instead of using an NMI to drive this, that I just have a tight loop that does both, and interleaves this with doing the visualisation. The goal here is not something that is perfect, but rather, something that demonstrates that we can produce and receive a signal.
As I was thinking about it overnight, it occurred to me that I might just be able to use the SID chip implementations in the MEGA65 to produce a tone at the target frequency. This would avoid all the CPU timing complications. The catch is that the SID doesn't do pure sine waves, but only triangle, saw-tooth and square wave waveforms.
The problem here is that this means that it will introduce harmonics. I don't know if the harmonics will only be at higher frequencies -- and thus not audible -- or whether there will also be audible harmonics on the lower end. I'm not a signal processing expert, but my intuition and little bit of radio experience tells me that we can expect at least hetrodyning from reflections. That is, the signal will mix with its reflections, and this will produce the sum and difference of each frequency present in the signal. If any of those harmonics are loud enough, it could produce an audible artefact. The good news is that this is, by definition, testable. So I can live with that.
So, coming back to producing tones with the SID chip, I need to find out what the maximum frequency that the SID can produce is. Fortunately I have done a bit of mucking about inside the VHDL SID implementation we are using, so I know, for example, that it internally uses a clock in the 10s of MHz to generate everything, so tens of KHz shouldn't be a problem. Also, the SID chip's frequency generation formulae are well known, so we should just be able to work out the correct register settings, and again, just try to produce something.
Let's start with working out the register settings... and here we hit a snag: Although the internals of the SID chip are capable of much higher frequencies, the registers for the frequency generators can't go above about 4KHz, because the 16-bit frequency values don't offer enough dynamic range.
Back to the drawing board then. The 16-bit digital sample registers on the MEGA65 will be much more effect, and allow us to generate close to a sine-wave, as previously discussed. The only problem is feeding them fast enough. It might be possible to make a really tight CPU loop to do this, but it will be a bit of a pain for playing the tone while also processing the incoming signal.
What I am thinking for a solution to this is to add Amiga-style intelligent audio DMA logic, so that I can point the machine to a sample table, and have it play the sine curve over and over at arbitrary sample frequencies. It will also be handy for the MEGA65 retro-computer in any case. For this, we will need the following information for each channel:
1. Base address of the sample data (28-bit address)
2. Length of the sample (16-bit number of samples)
3. Sample frequency
4. Volume level (maybe)
5. Flag to select 4, 8 or 16-bit samples (maybe)
Those last two are maybes, because they aren't strictly required, but will give more flexible output and make more effective use of memory, respectively.
The next question is where to locate them in the system. Unlike the Amiga, the MEGA65 has the CPU always being the bus-master. This means that it will end up in the CPU one way or another. There is a DMA controller in the CPU already, and that could be hacked to provide the means of setting things up.
It also actually reminds me of another big problem for the MEGA65 at least: DMA jobs on the MEGA65 are not interruptable. This means that if I implement this sample playback method, the sound will pause whenever a DMA job is running. With a bit of clever pre-fetch and buffering, I can probably hide this problem for all but the longest running of DMA jobs. Given a CPU frequency of ~40MHz, and the maximum 64KB DMA copy requiring ~130Kcycles (or ~260Kcycles for swaps, when they are implemented), that corresponds to about 7 milliseconds. If the programmer were disciplined, and broke the DMA jobs down into 1KB pieces, then we can get that figure down by close to two orders of magnitude, to ~0.1 milliseconds, which corresponds to a sample frequency >8KHz. That sounds like a reasonable situation for now, and I can always make the DMA jobs interruptable by the audio data fetch sometime down the track.
Anyway, all of the above means that a buffer of even just one sample per channel should allow decent audio, even with DMAs running. If I make the buffer even just 4 or 8 samples deep, then much higher frequency audio should be possible -- allowing us to produce ultra-sound while running any little DMA jobs that we might need in the test programmes I will need to write.
Since I've just talked myself into using the DMA controller, I might just make the control for this via unused extended DMA job options, rather than having more memory mapped registers. It also means that setting up a particular sample will be easier in practice, because you will just be able to trigger the relevant audio DMA job, rather than having to stuff a pile of registers. I'm liking this plan even more... except it will be a pain for freezing. So I'll have to find some spare memory-mapped register space, after all. Oh well, it sounded like a great idea.
Time to get cracking: First I need to setup the data structures, and create the memory mapped registers to access them. Those are $D720-$D75F, with $10 registers per channel. To that I have added all the behind-the-scenes plumbing that does the sample fetching, volume control and mixing. The whole setup is not over complicated, but simulation is still the best way to ensure correct behaviour. And that's where the problem has arisen...
When I simulate it, GHDL is not incrementing various counters in this new stuff correctly. This seems to be because it thinks that lines are being cross-driven. The trouble is, all the inputs to the calculations don't seem to have any undefined or other invalid values. This led to a whole rabbit-hole of investigation that took several full days to explore, trying to get backtraces working in GHDL, so that I can see where the meta-values are being produced, as well as dealing with GHDL crashing during compiling the code for synthesis, among others.
After various adventures, I got GHDL built using the GCC back-end with at least rudimentary back-trace support. So now I am working through the process of finding and eliminating the meta-value problems, so that I can hopefully get to the one that is causing my simulation problems. Simultaneously, I have been making small incremental changes to the VHDL and synthesising, to progressively inch towards making the audio DMA stuff actually work. The whole idea of simulation was to avoid this slow process, but here we are with both both running neck and neck, as to which will yield results first.
One thing I realised I needed to add, is a mechanism to prevent the audio DMA from hogging the bus. To do this, I have implemented a hold-off timer, that prevents any two audio DMA cycles occurring with less than 8 cycles between them. Combined with the fact that the DMA cycles cannot interrupt a CPU instruction, this should allow for reasonable processor performance, even when the DMA rate is set reasonably high. Also, it means that we should be able to get around 2 million audio DMA operations per second. Since we need only one audio DMA channel for the ultrasound, this should be plenty.
Well, that all took longer than intended, and I'm not yet sure that it is all 100% correct. There are some niggling CPU timing problems, that mean that the audio DMA only works reliably when the CPU is set to 40MHz, and not in the Hypervisor. More precisely, it probably will get upset if the CPU is running in anything that is not the main memory. Anyway, for now, those are reasonable limitations that I can work within.
The audio DMA system works by adding a 24-bit fixed-point fractional increment to the sample counter. When it reaches 1, then its time for a new sample. This means that the sample rate will be CPU CLOCK SPEED * FRACTION. If for simplicity, we call the fraction a simple 24 bit integer, and substituting the CPU speed of 40.5MHz in, we get:
SAMPLE RATE = 40,500,000 * (SPEED / 2^24)
And thus by rearranging, we can find the SPEED value required for any particular sample rate:
SPEED / (2^24) = SAMPLE RATE / 40,500,000
SPEED = SAMPLE RATE * (2^24) / 40,500,000
SPEED = SAMPLE RATE * 0.414252
This means that we can achieve sample rates all the way up to the CPU clock speed, and down to about 3Hz. In practice, it is limited to around 1 - 2MHz due to the bus saturation limit, and also because the audio cross-bar mixer effectively places an upper-limit on the sample rate that will come out the speaker. If the audio cross-bar is limiting the frequency too much, we can make a bypass for that, so that we can increase our upper frequency limit.
For a little test, I have made a 16-entry 8-bit Sine table, so that I can produce a pure tone for calibrating frequency and testing the volume at varying ultrasonic frequencies. It sounds generally ok, but even I can hear that it isn't a pure tone. So I might increase the sample count and go to 16-bit samples, so that it sounds better.
Well, that didn't work. I even tried it on the MEGAphone prototype, in case it was the audio output circuitry. However, fiddling around, I did discover something important: The distortion changes based on the code the CPU is running. I even made a little loop that confirmed that the opcode of an instruction is ending up in the audio stream, by changing the bytes I am playing in the sample loop, and finding the silent point occurred when the bytes all matched the opcode of the loop. More investigation reveals that it isn't just opcode bytes that can show up, but seemingly any CPU memory access. Also, I was seeing that just occassionally the CPU would pick up a byte from the audio DMA data, and jump off into lala land as a result.
This is rather annoying and a bit worrying, as it means that there is a bigger problem with bus timing than I had expected. I already knew that I had to be careful with not allowing the audio DMAs in hypervisor mode, and only at 40MHz because of funny business. And it now seems that this problems are much more significant than I had hoped. But because they don't show up in simulation, they are a bit of a pain to track down.
What I might do here is a bit of a pragmatic solution, and make the bus wait an extra cycle when starting an audio DMA so that the value we want REALLY shows up, and then allow the bus to settle for a cycle back on whatever the CPU was asking for before the DMA. Another slightly more elegant solution would be to use the dead read wait states in the CPU. But that is rather more complex to implement.
Taking a look at the synthesis logs, it looks like the audio DMA stuff has pushed the tolerance for timing closure on the memory controller in the CPU out the window -- which would explain just about everything. The trick is how to simplify things back down, so that the logic becomes shallow enough again to get closure.
It looks like it might be easier to use the read wait-state cycles after all. Those were all reading from address $000002, so they were easy to find. I've now gone through and refactored all the audio DMA scheduling into a more generic "background DMA" framework, where the sample LSB and MSB reads for the four channels are considered 8 separate DMA targets. It also means we can add other interesting background DMA actions in the future.
For not the first time in this adventure, I have had something that ALMOST works, but not quite. Sometimes it would simulate, but not synthesise due to multiple drivers, or there would be funny corner-cases where it wouldn't correctly realise when the shadow RAM was reading the correct location for the background DMA activitity. The whole interplay for reading the shadow RAM with effectively zero waitstates is just a pain, but we have to work with it for now.
I think I now have it so that whenever the shadow RAM is not being used, that the CPU does a background DMA read, and correctly latches the data into the audio registers. While I was doing all that, I also overhauled the audio mixer to use signed samples the whole way through, so that mixing the audio can be done more easily, and without introducing DC biases like unsigned samples do.
It now runs under simulation, with the background DMA reads happening when the CPU reads from IO, or has a wait-state: Basically it makes the background DMA activity the default, and it is only if a different address is presented to the shadow RAM bus that it does something else. So the moment of proof comes now, while I wait for it to synthesise again...
And again. And a few times after that. But I now finally have a nicely working DMA audio engine. In fact, its now good enough that it can play Amiga MOD files with a crude little tracker that I wrote to test it. Which actually works quite nicely. It can already play a reasonable variety of MOD files, but doesn't yet support most of the MOD effects -- only tempo and instrument volume.
But it does already support repeating samples, too, since I need that for the ultrasound testing, because it would be a pain to have to keep feeding sample data in. Instead, I can just play a 32-byte sine wave loop indefinitely. The result is this:
From there, I have been working on a little test programme that just plays a continuous sine wave tone, with variable frequency. The Audio DMA can, in theory, play a sample every tick on the 40.5MHz CPU clock. However, in practice it is limited to about 1/16th of that, i.e., about 2.53MHz. Still not bad. Our sine wave sample is 32 bytes long, so that means we have a maximum of around 80KHz. This is comfortably well above our target range of 20 - 30KHz -- and remember that this is not the frequency at which it can play a horrible square wave, it is the frequency at which it can play a pretty nice sine wave -- and all this without the CPU needing to do anything once we set it going. So I'm pretty happy that we have the audio sub-system in place that will let us produce ultrasonic frequencies.
Now, back to that test programme: It plays the 32-byte sine wave in an infinite loop, and lets us vary the frequency and volume. It also shows a nice little oscilloscope display of what it thinks it is playing:
Here we can see it set to ~40KHz, above what we need, and the 40MHz 8-bit CPU is doing a fine job of displaying this in real-time. It is currently fixed with respect to the sweep time, with the 256 pixel wide display corresponding to about 130 microseconds as a natural consequence of the tight sample reading loop. It took a little bit of fiddling to get the programme right, though:
First, I had to synchronise the sweep to always start at the beginning of a loop of the sample. This helps to hold the display with a fairly steady phase.
Second, I was using DMA to update the display with a double-buffered arrangement I had made for another programme. However, that can't be used here, because foreground DMAs, such as the double-buffer copying, cause the audio DMA in the background to pause. This was causing quite horrible audio artefacts as the tone would be interrupted tens of times per second for a number of milli-seconds. Eliminating the double-buffer and all other foreground DMAs fixed that problem. This is one of the things about this audio sub-system that leave it clearly in the 8-bit home computer class, and not in the multimedia PC class where the Amiga lives: There is no prioritised media DMA slots to ensure stable audio at all times. Demo writers thus get to have fun working out how they can do parts with cool DMA effects *and* still have nice digital audio playing in the background.
Third, even when I fixed those problems, there is still a funny artefact where there are spikes in the audio playback. There are some hints that this might be a glitch when the next sample begins to play. However the audio hardware is so fast -- driving the PDM at 40.5MHz, resulting in ~25ns intervals, makes it hard to capture this reliably on my oscilloscope. This causes wave-forms that look like this:
This problem is manageable, however, as the sine wave is still otherwise intact, and produces an acoustically decent tone. The glitches do produce some audible artefacts when driving ultrasonic frequencies, but this is all at tolerable levels for the test. So I'll worry about fixing that later, unless it does turn out to be problematic.
The next step is to add the reading of the microphone data. Hopefully this will go smoothly, as I have already established that the microphone is sensitive well past 20KHz, and according to the data-sheet, all the way up to the maximum 79KHz that we would realistically generate.
Getting the code running on the MEGAphone prototype was easy, as it is fully compatible with the desktop MEGA65 that I have been developing it on. The only trick was I had to remember how to control the amplifier on the MEGAphone for the speakers so that I can get the volume loud enough for testing. This is controlled by $FFD7035 and $FFD7036, where $00 = +24dB, $40 = 0dB and $60 = -24dB and $FF = mute. It was set to $60 by default and was way too quiet.
I did some initial testing with $00 (+24dB) and it seems to work, although I think it might send the amplifier into over-current shutdown after a while. That would be ok for this application, as we only need a very low duty-cycle. What is more of a problem is that at +24dB there are a lot of audible artefacts which I will need to investigate. Once the kids have gone to sleep I'll start experimenting again to see if +0dB is loud enough to be detected at a decent range.
So, in terms of initial tests, I'm initially testing only at very close range, and having to deal with some distortion caused by a bug in the audio DMA subsystem that I have yet to figure out: Basically if the CPU is running, then the waveform is distorted. I'll likely deal with that by implementing a special "play sine wave" mode for the audio subsystem, so that we can side-step the whole DMA thing altogether. But its still good enough that we can work out where the resonances are that are good enough to receive anything. So:
There is a peak between about 20.0KHz and 20.2KHz, that ramps up more slowly on the low side, and falls off quite quickly on the high-side. Then there are also a few peaks in the 25KHz to 30KHz range, but they are lower than the 20.1KHz centred peak. Above 30KHz I haven't seen any signs of life as yet. But this DMA distortion problem is getting worse, the higher the frequency, so I think I need to implement the "play pure sine wave" function, and then test it again. But I must say: I'm pleasantly surprised that the speak is in fact producing any energy at all above 20KHz. Whether it has enough range, and whether it can do so with quiet enough audible artefacts remains an open question, though.
I implemented the sine table ROM to avoid the distortion. There are still audible artefacts, presumably because the MEGA65's audio output is PDM, i.e., lots of 1s and 0s rather than a true sine wave. As side-effect of fixing this is that the peak response shifted down from ~20.1KHz to around 18KHz. There is some response up around 26KHz-30KHz, but it is quite weak.
It looks like the next step required will be somehow filter the signal, so that we get closer to a true sine curve. The data-sheet for the amplifier suggests using ferite beads on both leads to the speaker, and a 470uF capacitor to GND on the high-side. At that point, though, we cease to be working with the existing MEGA65 hardware without modification, and we are already well past the scope of this work unit.
What I will still do, is attempt to more adequately quantify the range of ultrasonic frequencies that respond, and what the fall-off looks like, so that we can identify the likely usable frequency range for communications. Minor speed-bump, though, in the form of the speaker lead breaking:
Fixed that. Next step was to improve the ultrasound test progamme to show some kind of frequency domain break-down. An FFT would be idea, but a fair bit of work to produce, when I just want to be able to see if there is noticeable energy at a frequency. So I made a simple programme that superimposes the sample train over itself at all possible time deltas, and then measures the energy of the result. If the delay corresponds to a periodicity of the signal, then it will result in a higher amplitude signal. I added a bit of filtering to it to deal with harmonics etc, and give or take a few wrinkles, it produces fairly clean peaks for the frequencies at which energy is present:
The read peaks correspond to the periods at which energy is present, i.e., further left is higher frequency, and further right is lower frequency. The very tall line-like peak on the left edge is an artefact of measuring energy by superimposing waves and looking fo reinforcement, as it will basically pick up when adjacent samples don't zero-cross. Thus it should be ignored. But we can see the next peak corresponds to 1 cycle of the ~76KHz resonance of the microphone. The other peaks come and go a fair bit, and are less robust, and may correspond to some weaker lower frequency enveloping of the 76KHz resonance.
So I think we are finally set to try various frequencies and measure at which frequencies we reliably see energy, and to delineate that set of frequencies as those which are potentially usable for ultrasound communications on the existing MEGAphone hardware, and thus to give us sufficient characterisation of the available channel bandwidth.
I'll work with the amplifier set to $40, which is some way off the loudest setting, and the speaker leaning against the MEGAphone prototype about 1cm away from the phone. If we can't pick up a signal under those conditions, then we'll assume it won't be usable. But if we can, then we can repeat the test with some increased distance and power, and see what kind of range is possible. To support this, I have improved my test programmes to work out the period of the sample frequency requested, and to calculate a running average of the estimated power at that frequency. In this way, I have an objective and quantitative means of comparing the power at different frequencies, even if the units are undefined because of the estimation process. The display with this looks like this:
The light blue vertical line is the period we are looking for. This corresponds to exactly one full wave of the tone being played (visualised by the white points). The received return via the microphone is the yellow points, and the red line is the power at each period.
So we can see visually that there is some power at the target frequency here visually, as there is an observable periodicity in the yellow waveform, which is then reflected by the red peak at that point. The running average is then calculated and displayed in the lighter red coloured text. For each sampling, I will collect 63 samples. I'll also do a quick variance check by doing three successive collections with a single setting, so that we can get an estimate of how reliable the readings are. It was quite pleasing to be able to implement all of this directly on the MEGAphone's 40MHz 8-bit CPU.
Okay, so having batched a bunch of runs at different frequencies, we find the following:
1. Below ~19.4KHz the waveform is very loud.
2. Between ~19.4KHz and 19.7KHz it's not quite as loud, but still very loud, perhaps 1/2 as loud as just below 19.4KHz.
3. From 19.8KHz to 20.4KHz it's about 1/2 as loud again, with relatively flat response.
4. From 20.4KHz to about 21.4KHz it is about as loud as between 19.4KHz and 19.7KHz again.
5. From 21.4KHz it drops off very sharply, with no discernible signal beyond 21.7KHz.
So assuming we want to keep high enough frequency to not annoy people, there is probably about 1.4KHz centred on 20.7KHz that would be usable. As tempting as it would be from a link-margin perspective, dropping down to below 19.4KHz is probably not feasible.
Increasing the amplifier level from 0dB to +12dB yielded a
discernible waveform at distances of around 30 - 40 cm at 19.4KHz --
i.e., where the performance of the system is very good. The amplifier
can go to +24dB, which would thus be expected to perhaps deliver the
~1.5m range, but at a power consumption during transmission of greater
than 2W. It seems rather unlikely that 1.5m range would be obtainable in practice over the whole band, although somewhat shorter range may well be possible, or it may be possible to increase the amplifier power further.
But whatever the frequency and amplifier arrangement, some kind of filtering is going to be required, to prevent the audible artefacts that are present across the whole frequency range tested.
That said, we have clearly proven that the microphones we are using are sensitive to ultrasound. Thus it could be possible to use a smaller speaker with improved ultrasonic properties to further boost performance.
But as I reflect on all this, it seems to me that there are a bunch of problems, that together make me a little sceptical about the utility of such a system in practice:
1. Transmit power of 2W or more is likely required to produce a useful range over a wide-enough ultrasonic band. Even at a low duty-cycle, this will create a significant power consumption.
2. The speakers we use, and most ultrasonic speakers, are rather directional, making detection of proximity rather unreliable. It was quite fiddly to get the results described above, demonstrating that this is not just a theoretical problem.
3. Point (2) is made worse if you have the device in a bag or pocket.
4. The audible artefacts are REALLY annoying.
5. Some people can likely hear up around 20KHz, and even the occasional short chirp of a packet is going to be very annoying to listen to.
6. Bluetooth is already widely available, requires no new hardware, and for all its problems, has much better performance than I have been able to observe here. In particular, the lack of risk of audible artefacts seems compelling in the circumstances.
7. The need for contact tracing apps seems to have somewhat dissipated, at least for the time being.
However, as a further communications channel between devices when faced with a hostile RF environment, it would seem to have potential. In that case, it would likely be possible to mask the audible artefacts by, for example, playing music or other sound while the ultrasonic transfer occurs. In this context the difficulty of creating an ultrasonic signal that can travel great distances becomes a strength, because it means that an effective jamming effort would require considerable proximity. In contrast, 2.4GHz Bluetooth or Wifi are rather easy to jam from a distance.
Thus, while this wasn't the primary objective of this investigation, it has revealed that such near-ultrasonic communications is quite possible using rather conventional components found in smart-phones and similar devices. It also means that from a privacy perspective we must take care, as it is similarly possible for devices to communicate via ultrasound without a user's knowledge, e.g., to exfiltrate data across air-gaps.
Goodness me, it's been a long time that we have been working on the MEGA65 nowhelp. But we are finally at the super-exciting end of things where things are starting to happen more quickly. And that includes this week, when pre-ordering of the DevKits will open over at Trenz Electronic.
So let's talk about what the DevKits are, and what the price will be.
But before I continue, I just want to be careful to explain that the pricing of the DevKits is not representative of what the price of the final machine will be. There are a number of reasons for this:
First up, at this early stage of things, we don't have the price of the various components optimised. We are buying smaller quantities, and learning how to do everything thing right. That all translates to increased costs for the DevKits. We hope to be able to sell the final machines for less than it costs us to manufacture the parts for a DevKit, for example.
Second, the DevKit release is partly to help early adopters get hold of the machine and start writing software, documentation and other goodies for the community, and partly to provide us the cash-flow to get the production machines out ready as soon as possible. This means covering the cost of designing and printing the packaging, user guide and other goodies. While we are already well advanced on some of that, there are some very real costs that we need to cover.
Third, somewhat counter-intuitively, the acylic cases of the DevKits are probably more expensive than what the injection moulded cases of the production machines will cost. This all comes back to the wonder of injection moulding: You pay a fortune up front, and then you can produce the best cases at the best price for a long time after. That's where we are aiming to be for the production machines.
All up, expect that the DevKits will be quite noticeably more expensive than the production machines. Also, remember that the MEGA65 is an open-source computer being produced on a non-profit basis: None of the MEGA65 team earn anything from the sales. It all goes to cover costs and support the completion of the machine.
But let's get back to talking about the DevKits: One of our big goals is to increase the number of people who can develop on, and contribute to the MEGA65. We see this as a once-in-a-generation opportunity to help shape and be part of the story of The Last 8-bit Computer that never was, and now will be. There may be other 8-bit systems in the future, but as the spiritual successor and completion of the C65, we think the MEGA65 has a pretty special role to play -- and we'd love to have more folks help us make it as awesome and exciting as we can.
We want it to be a machine with a variety of software and good compatibility and rock-solid performance when it is ready to arrive under people's Christmas trees, birthday present piles and Retro Rooms, so that we can recreate that "Christmas 1982" feeling one more time for the community. To achieve this, we need as many people to contribute in a variety of ways, whether helping with documentation, C65-fixing existing software, writing new games, programmes and tools, or contibuting to the VHDL and operating system software of the MEGA65 itself.
It's a radically different model to most computers around today, that are more like mountains upon which we gaze, or perhaps at most seek to climb. But we want the New Zealand model, where geography isn't just something you look at, but is rather more of a participatory sport. So to with the MEGA65, it's by participating in the story that you have the most fun, and can share the most joy with the community. Come. Be part of the story with us.
Frequently Asked Questions
Q: What's a DevKit? A: A development kit aimed at developers so they can start coding software for the machine and even help shaping the final product before it is released.
Q: Why does it look so different? A: The case is made in a way it can be produced in small batches before injection moulds are finished. Its transparency helps finding out if the smoke stays inside the chips.
Q: I am a collector not a developer. A: The DevKits have laser-engraved Logos and serial numbers to make them unique. DevKits usually are great collector's items.
Q: I only want to play with it! A: The DevKit is like a "real" MEGA65 only in a preliminary form. You might encounter hiccups but you can always (soft-)update it.
Q: I do not like the floppy. A: The DevKits come as (hence the name) kits which include a refurbished floppy drive. Feel free to leave that out and donate it to us.
Q: What will it cost? A: DevKits are always more expensive than mass-produced machines, they also get strong support from the makers. The MEGA65 DevKit comes with a price tag of EUR 999 which is in fact very low considering the cost of the components, support and general preparations required for this initial production run of machines, as well as our costs of getting the final machines ready for release. The final machines will benefit from these things, and improved economies of scale, which will allow it to be released at a lower price.
Q: Can you build it for me? A: It's really easy to build, usually under an hour, maybe a bit more if you are clumsy. If you do not dare to build it yourself please ask in our support forum or via the other communication channels you will get access to. There are many nice people around!
Q: I am more interested in the final MEGA65 and not really a developer, how can I support and improve the development of the final machine? A: Please buy a DevKit and lend/donate it to a talented developer!
Q: When can I buy it? A: From tomorrow on, but do not wait too long!
Q: I am a blessed developer and want to sacrifice all my time but I do not have any money! A: Please talk to us about support!
Q: If I buy a DevKit, can I transfer the PCB and keyboard into a MEGA65 case later? Can I even 3D-print my own MEGA65 case? A: Most probably yes! But we can’t guarantee it.
