MEGAphone R4 PCB Bring-up Part 1

I finally have the new MEGAphone R4 PCB in my hands, and just a few weeks left to do the hardware bring-up to meet the final milestone of the NLnet grant supporting this -- so it's all hands on deck now to get this done.

First up, here is what the R4 PCB looks like after Goran did initial testing and debugging a few issues with the schematic.

First, we have the front-side, where the buttons, speaker, thumbwheels, and of course, LCD panel will go.  Down the right hand side you might be able to see all the power isolation switches.  There are 6 of those, that we will explain later as we go through, which can cut power hard to most subsystems.

You will see that Goran has had to make a few modifications based on his initial testing:

Those will all get fixed in the R4A PCB layout.

Finally, we have the rear-side, with most of the interesting bits:

Taking a tour clock-wise around this board from the top-left corner, we have:

1. The HDMI connector (grey with the orange bit of tape) for digital video output.

2. Outline showing where a Raspberry Pi Compute Module 4 can be attached.  This is so that if you want to run Android or other "big" things, you can safely do that in secure compartment, i.e., the Pi, that has no control over the rest of the system.  The display from the Pi will be combined using the MEGA65's GenLock functionality, to allow the MEGA65 display to overlay over the top of the Pi's display, complete with transparency.  That last bit of the red channel of the C65's palette will finally see it's use!

3. To the right of that is the connector to hold an OrangeCrab FPGA module.  This module is an over-kill for what it needs to do, but with COVID supply chain dramas, we had to run with what we could get.  The Crab listens to the serial ports from the cellular modems and other communications bays, and can then activate power to the main FPGA to wake it up if something interesting happens. Like someone calls you or sends you an SMS.  This is super important for keeping the idle power consumption of this FPGA-powered phone down low enough to give it a long battery life.

4. To the right of that is an 18-pin header that is designed to accept an ESP32 to provide WiFi and Bluetooth support to the MEGAphone.

5. Then we have that port that every mobile phone should include: A standard Atari/Commodore-compatible joystick port! No more being limited to touch based games for us, nor to single-player games when someone else is using the D-pad and press-buttons on the front. You will be able to connect your favourite Commodore 64 joystick for authentic game play while under way.  Also handy to use as a clicker when using the MEGAphone to make presentations using the digitial video output.

6. Then because we are not Apple, we include a 3.5mm headphone/microphone jack.

7. In the top-right corner we have the MEGA65-compatible power pack input. This can take from 6V to about 20V, and doesn't care if it is a mains powered power pack, or a solar panel, with its integrated Maximum Power Point Tracker (MPPT). This is to support the integrated solar panel that we plan to put on the rear of the MEGAphone, potentially allowing infinite battery life. In the future, I would like to also add a USB-C power input, not just because the EU requires it, but also because it is common and convenient. But that might have to wait for the R5 PCB.

8. Coming down the right hand side, we have the big green connector for the battery.  For testing I am using a 10A 3.2V LiFePO4 prismatic cell, which is also the cell type I intend to include in the prototype MEGAphone R4 units that I will build up. This is what the cell looks like:

9. Below that we can see the two silver SIM / SD Card holders. One SIM card for each of the two cellular modem bays that live in this corner of the PCB.  Only one of the SD card slots is connected to the main FPGA.

10. Very near the middle of the PCB, there are two 16-pin connectors. These are designed to accept RFD900 / 3D Radio style UHF packet radios or other utility modules that folks might come up with.  Having a place for these is to allow the MEGAphone to one day act as a completely stand-alone mobile communications device, being both energy independent (thanks to the power efficiency, large battery and integrated solar cell) and communications infrastructure independent through mesh/ad-hoc wireless networking.  

11. Then along the left hand side we have the two black headers for the main FPGA module, a Trenz Electronics TE0725.

There are other various small bits hiding around the board which we will meet as we progressively bring up the various sub-systems. But that's enough to get us started.

The first step is getting the board to power up, and to be able to talk to the OrangeCrab FPGA board, since that controls the power supply to every other module on the board.  This uses a Lattice ECP5 FPGA, which has the nice advantage of being fully supported by the yosys open-source FPGA development tool chain. Not only is it free (in both senses), but it's also really fast.  The Xilinx Vivado suite takes well over a minute on my i7 to synthesise the simplest design. With yosys, its less than 4 seconds. This really helps for rapid development.

Speaking of which, one nuisance of the OrangeCrab, is that to flash it via the USB port, you have to drop it into "DFU mode". This is done by powering it off, and then holding down the single press-button on the board while you power it back on.  For testing the MEGAphone, this would mean disconnecting both the USB cable (which carries power) and also the battery and any power pack.  This is a lot of fiddling, and could wear out connectors over time, as well as just being a pain.  The PCB design also requires the battery to be present for the OrangeCrab to boot up, so I can't even optimise it by just having the power pack, but not battery connected.  It can run from just USB power, but then the other sub-systems of the phone won't be able to be powered up.

On Xilinx FPGAs, there is a way you can tell the FPGA to "reboot", which we use to support multiple "cores" on the MEGA65.  As the DFU Bootloader is just a "core" on the OrangeCrab we should be able to do the same. After quite a bit of hunting, I discovered that the SB_WARMBOOT primitive is the apparent way to do this on Lattice FPGAs.  However my Verilog is quite scratchy, and this is what yosys prefers over VHDL, and I couldn't get it to recognise this primitive.  So with Goran's help I discovered that while the PROGRAMN pin of the OrangeCrab, which does a hard reset of the FPGA, isn't connected to any other pin to make it easy to drive, it can still be driven from FPGA logic internally.  So with a single line of Verilog, I was able to make it so that if you press the push button at any time, it will reboot the FPGA into the DFU mode:

   // Reset when btn0 is pressed

   assign rst_n = usr_btn;   

This now means that I can super-conveniently just press that button when I want to re-flash the OrangeCrab FPGA with a new bitstream.  So time to load it onto the board, hookup the battery, provide some DC on the power jack, and get moving.

We have LEDs on the crab lit, and the green LED near the joystick port indicates that the board is charging the battery (quite slowly, because its only powered by the micro-USB of the Crab right now, which I think is limited to the old 500mW USB 1.1 standard).  

I can see the voltage on the battery slowly climb, confirming that the charge circuit is working.  If I disconnect the battery, the charging LED goes out, and the fully charged LED below it lights instead.  So all up, the battery charger is doing what it should. 

There is one issue with it that needs to be cared for, though, which is that as LiIon charger, it will aim for 4.2V, which is the absolute upper limit of the LiFePO4 chemistry.  The 1% tolerance of this could take it over the edge.  So we will need some sort of means to disable the charger before it over-charges the battery.  The charger is a CN3791, which doesn't have any mechanism to disable the charger nor to tweak the upper voltage limit.

We do feed the CN3791 via a LM74610-Q1 reverse-polarity protector. It might be possible to kludge that to disconnect the battery from the charger if the battery voltage climbs to high, but that would also disconnect the battery from the whole device, which is rather sub-optimal, shall we say, to treat a full battery like a completely empty one...

I'll have to have a think about how we solve this. It might be possible to add a diode to cause a small voltage drop at the right place in the circuit. Or better, we find a CN3791 equivalent that is made for LiFePO4 characteristics. Like the CN3801 from the same manufacturer, that is identical, except that it is for 3.625V, not 4.2V max. So I'll order a couple of those, and switch it out when they arrive.  Sounds like a perfect solution. It's only the paying $20 for postage on a $0.50 part that is making me reluctant. But I've got over myself and ordered some. It will be nice to put it on, and then not worry about frying my nice LiFePO4 batteries.

I'll try charging from a solar panel later, when I have time to hook up a lead on the panel. Spring has only just started here, so I'll also have to wait for a suitable patch of sunshine. It's a lower priority than getting the rest of the system powering up.

So, on to getting other things to power up, we have a direct 5V enable line, that is used to provide power to the microphones. This is under the control of the Crab, and was easy to confirm working. But the rest of the sub-systems have their power enable lines controlled via an I2C IO expander.  The Crab needs to talk I2C to the expander to set the appropriate lines to output, and set them high.

I've done plenty of I2C communications before, although I do always find it frustrating to get rock-solid.  But I can just lift the I2C controller from the MEGA65, if I trust the vhdl2vl VHDL to Verilog translator. Or I can use the I2C controller in Verilog that Goran has already put in for testing.  I'll give that a go in the first instance, and use my oscilloscope to probe that things are doing what they should.

The IO expanders are PCA9535s. This is the connections for the one we care about right now:

 

There is an error in the schematic, in that we have pin 3 labeled RESET when it should be A2. But that's not a big problem. I've filed an issue for us to fix that.  The main thing is we can work out the I2C address for this device connected this way: The top 4 bits are fixed to 0100, then we have 100 for A2--A0, so the 7-bit address will be $24, i.e., $48 for write and $49 for read.

I have also just noticed that this IO expander is powered from VCC_FPGA, which can get turned off. It should instead be powered from the same power source as the Crab, so that it stays on.  I've noted that in an issue as well.

Except it doesn't seem to be quite so simple... With the crab removed and only the battery, I see 3.46V on VCC pin of U14, even though the battery is showing 3.26V. Unless we have a "negative resistor" on the board, something funny is going on.

First, let's trace VCC_FPGA:

 

We can see that it comes from Power Supply PS1, and that the PS_ENABLE signal that enables it, that this depends on the position of switch S4, which is the 3rd switch from the top on the right side of the board: If the switch is in one position, then VCC_FPGA is definitely on. If it is in the other position, then the FPGA_Power_Off signal can be used to disable VCC_FPGA.

Most of the power cut switches disable power to a subsystem, but we made this one the other way around, so that we couldn't end up with a dead board in case the SN74HC74 didn't behave the way we expected.  This is because we are using it in a slightly odd way: By pulling /PRE high, we are requiring /INT to go low to activate power to the FPGA.  On the other hand, we can use the FPGA_Power_Off signal to reset the flip-flop, thus cutting power to the FPGA. 

FPGA_Power_Off is a direct output from the Crab, effectively allowing the Crab to ask for the main FPGA to be turned off.  The simplest source of the /INT signal is the second button on S6, i.e., the right-hand side of the two press buttons on the bottom of the device.  This gives us a nice hardware power-on switch.  So that all makes sense -- its just the mystery of why we connected the IO expanders that control other power systems to VCC_FPGA, as this will require the main FPGA to be powered on (and consuming power) for us to be able to turn off the other sub-systems. 

With that issue filed, let's get back to getting the I2C communications happening, so that we can test enabling and disabling the various power sub-systems.  The key enabling step here is that I need to be able to monitor the I2C bus with my oscilloscope, so that I can see what is being communicated.  The only place I can really get to these pins is on the connector of the OrangeCrab.  The question is how I can get my probes to stay on those pins, as I have only the tail of the hole-through header, which aren't really conducive to holding oscilloscope probes.  If I had thought ahead, I should have just had a 2-pin header with the SCL and SDA lines on it, so that I could do this part more easily.

One of my colleagues at work makes beautiful masterpieces from clothes pegs to hold probes in all manner of strange positions. I might be able to do the same here.  

And the proof of the pudding: I can see the I2C traffic on my oscilloscope and I can reach to press the reset button the crab board without much risk of disturbing the setup.

 

So now to overhaul the I2C traffic generation so that it queries and sets the registers of the IO expander.

I was able to fairly quickly get it to repeatedly attempt to read multiple registers. I can see the bytes are being acknowledged on the oscilloscope.  I am seeing the same two byte values repeatedly, even when I ask for more than two bytes. The values are 0x00 and 0x7F, which based on the schematic snipped above, means that the various EN_* lines to enable power supplies are all low, as is LCD_STBY and DTR_M1, and the non-connected bit 7 of the second port. This sounds at least plausible.

I know for some of these IO expanders that this is a feature, that allows you to reduce the latency when monitoring the registers that correspond to the inputs.  So I am having a read through the datasheet now, to see if that is the case.

Table 8-11 on page 22 of the datasheet seems to suggest that it does repeatedly just read a pair of registers, rather than sequencing through the 8 registers. It seems that this behaviour is controlled by whether or not the register address is supplied for the read.  Anyway, it explains what I am seeing, and also how I can work around it.

Next step is to make the sequence write to all 8 registers, so that I can set the state of the power rails. The power rails are:

Port 0 Bit 0 - Cellular Modem Bay 1
Port 0 Bit 1 - Cellular Modem Bay 2
Port 0 Bit 2 - Packet Radio Connectors (both)
Port 0 Bit 3 - ESP32 / WiFi Module slot
Port 0 Bit 4 - LCD Panel
Port 0 Bit 5 - Audio amplifier for the speakers
Port 1 Bit 5 - Audio amplifier for headphones

The other bits can all be safely set to zero for now.

For testing, I will make the loop set both ports to output, non-inverted, and change which power rails are being enabled every 0.5 seconds based on the counter loop.  This will let me easily check if they are all doing their thing.

To check if power rails are on, the MEGAphone has red LEDs on the front side that should illuminate when the rail is turned on, and the INDICATE button is pressed. That's the button to the left of the soft "power on" button.  However what I am seeing is that LED D2 is permanently on, whether or not I press the indicate button, and that nothing ever turns on when I press the illuminate button.

LED D2 staying on is okay, actually, because it is one of the LEDs that is under direct control of the Xilinx FPGA, which I don't have plugged in, so it is floating, which is allowing the FET for the LED to activate, and thus the LED is illuminated.

However, the other LEDs should turn on with activate.  Time to back-track and check if the port pins on U14 are being toggled... which they don't appear to be.

I think I will modify the I2C transaction chain to both write and then read the registers, so that I can see if the writes are taking effect or not. I can see the writes on the bus, but they aren't actually setting the registers. Actually, this reminds me that I once had a problem with the I2C stuff not finalising a write transaction problem -- the same thing could be happening here. Yes, that was indeed the problem.  

Now that I can write to a register, I will now rejig it to setup the registers correctly to allow control of the power rails as I originally planned. The registers are:

0 - Read Port 0
1 - Read Port 1
2 - Write to Port 0
3 - Write to Port 1
4 - Polarity inversion Port 0
5 - Polarity inversion Port 1
6 - Data Direction Port 0 (1s= inputs, 0s= outputs)
7 - Data Direction Port 0 (1s= inputs, 0s= outputs)

So we need to write to ports 2 through 7, putting setting the power rail control bits to output, and writing some sensible values into the others:

2 - Bits 0 -- 5 = power rail enables (I'll make those cycle through on a counter)
3 - Bit 5 = power rail enable for headphones amplifier (I'll also make that cycle)
4 - All zeroes to disable inversion of bits
5 - All zeroes to disable inversion of bits
6 - 0x3f = 1s in the bottom 6 bits that are power rail enables.
7- 0x20 = 1 in bit 5 for the power rail enable there.

Except that if I try to write to all the ports at the same time, it doesn't work.  It might be that writing to ports 6 and 7 overrides writing to the actual port. Anyway, I will have to break it up into several single register write transactions.  This isn't a big deal. It's just a little bit fiddly, because my I2C control state machine counts the number of times the i2c_busy flag advances, which means that at the end of a multiple action transaction that the state machine won't know to advance.

I'll start by confirming which register write is causing the write to the port 0 output register to fail. Hopefully I can set both registers 2 and 3 in one transaction -- which I can.  So let's see how many more I can do... Looks like writing to registers 4 and 5 stops it working.

Reading the datasheet there is this ambiguous description of writes:

"The eight registers within the PCA9535 are configured to operate as four register pairs. The four pairs are Input Ports, Output Ports, Polarity Inversions, and Configurations. After sending data to one register, the next data byte is sent to the other register in the pair (see Figure 8-8 and Figure 8-9). For example, if the first byte is sent to Output Port 1 (register 3), the next byte is stored in Output Port 0 (register 2).
There is no limitation on the number of data bytes sent in one write transmission. In this way, each 8-bit register may be updated independently of the other registers."

I think this means that you can only write to one register pair at a time, and that if you write more than 2 bytes in that transaction, that it just writes over the same two registers again and again. While a bit annoying, I can easily work around that.

It looks like I now have it all working, in that I can write to the registers and read back the updated port values. However, I am still not seeing the pins on the IO expander toggling.  But its bed time now, so I will have to investigate that tomorrow.  It's at least some progress, though.

Tomorrow has come, and I had an idea overnight as to what the problem would likely be:  I was clearly talking I2C to one of these IO expanders, but the pins on the one in question weren't toggling, so I checked whether I had the correct address, which was indeed wrong.  It should have been $24, as calculated earlier in this post, but for some reason in the code, I had $25 there.

Now I can toggle power rails, but there seems to be some funny effects going on.  To debug those, I really am going to need to have a serial interface to command the Crab to turn particular rails on and off, so that I can see what is happening.  Basically turning some rails on seems to be turning on or off others, possibly due to back-flow of current among the sub-systems.  But that can wait for The Next Blog-Post(tm)!