Open Hardware Ethernet Switch project, part 1

"I was tipped off by a friend to a batch of Kintex UltraScale+’s, specifically the XCKU5P, on AliExpress for a mere $55 each. He had tested one from the seller and they appeared to be legitimate, although likely salvaged/reballed from some scrapped equipment."

Must be nice to know enough to be sure that your sketchy hardware isn't backdoored. I cannot imagine buying random network hardware from a sketchy source. Though I'm sure that if I had the author's chops, I wouldn't be too worried.

More broadly, network hardware is where the keyest vulnerabilities are. Open source network hardware is interesting from that perspective. You do not want random bad actor open source contributors adding backdoors.

There are no blobs I'm aware of anywhere that are actually running in the system.

The STM32s have a small boot "ROM" burned into a write-protected region of flash but I have it jumpered so I boot from main user flash, not the bootloader.

I did a quick silicon overview of them (just out of curiosity... they came new from Digikey so I have no reason to believe they're fishy).

STM32H735: top metal only https://siliconpr0n.org/map/st/stm32h735/azonenberg_mz_mit20..., deeper dive planned but haven't had a chance

STM32L431: did a full teardown https://serd.es/2025/01/02/STM32L431-teardown.html

The 12-port PHYs are a fused-down version of a switch chip so there is a MIPS core on there, but to the best of my knowledge in the switch SKU it doesn't actually run (e.g. its RAM bus is NC and the IO power rail is grounded).

The FPGA is completely programmed from the bitstream I control. Tampering with a random resold FPGA to add some kind of backdoor is extraordinarily difficult and unlikely, it's the kind of thing you would see done as a targeted attack rather than just dumping FPGAs into the secondary market and hoping that a juicy intelligence target buys them rather than some guy tinkering around with open source projects.

And, if I ever get the slightest hint that there is something fishy about silicon I bought from a sketchy overseas source, I'm gonna take it into the lab at work and go to town. I do semiconductor reverse engineering at my day job and am the absolute last person anyone should try to sneak backdoored chips past. It's going to end up getting dissected inside a SEM and turn into an awesome talk at REcon or something.

While it would be slightly annoying to have my side projects disrupted, having a living breathing nation-state silicon backdoor show up in my lab would be a dream come true. I'd be ordering as many more chips as I could from the same seller and instrumenting it in every way possible, practicing on non-backdoored chips to make absolutely certain I didn't damage any of the spicy ones while studying the altered circuits, etc.

You can actually now get realtek rtl838x based ones that run openWRT. That's open firmware not open hardware, but good for most purposes.

The overall system will be powered by 48V DC on a 6-pin Molex Mini-Fit Jr connector, then stepped down to 12V by a 48->12V intermediate bus converter I've previously designed and characterized (https://serd.es/2024/10/15/Intermediate-bus-converter.html)

The 24-port line card has a 15W TDP overall (6.75W datasheet worst-case power consumption for each of the two 12-port PHYs, plus 1.5W added for power supply conversion losses and such). With my current bench setup (11/12 links up on PHY 0 and 2/12 on PHY 1, all linked up at gigabit but with PHY 1 not talking to the FPGA yet), they're happy but warm - 63.8 and 49.5C die temperature respectively, pulling a combined 7.173W (not counting power conversion losses) according to the internal sensors. PCB temperature is 37.2 to 49C at various measurement points.

The entire setup including the FPGA board, IBC, one of the two planned line cards, and some other glue components that won't be in the final switch is pulling 18W and change right now although the FPGA power consumption will go up as I build out more logic. I don't have measurements of just the FPGA's power consumption currently (I could put a current clamp on the 12V cable from the IBC I guess) but the PDU board does have I2C sensors that will measure consumption of the logic board and each line card separately; I just haven't written the firmware to read them yet. I also don't have FPGA temperature readings in the current firmware although last time I checked them via JTAG I had plenty of margin.

As of right now everything is happy with low-profile heatsinks (Wakefield-Vette 960-27-12-D-AB-0 on both the PHYs and FPGA) and passively cooled just sitting on my bench with no fans. I can go taller if increased power consumption or worse airflow in the future dictates, they're nowhere near the point of hitting the top of a 1U chassis.

My plan for the final system is to have air intakes somewhere on the sides and/or the front around the RJ45s and one or more fans exhausting out the back, details TBD based on more design and testing once I have better projections of the overall thermal load.

Very roughly my overall thermal/power budget is 15W per line card = 30W combined, plus no more than 20W for the logic board (probably quite a bit less) with all ports lit up and passing packets, for a total of 50W plus whatever the losses in the IBC are (it's roughly 94% efficient at this load based on previous testing). This is comfortably below the 72W output limit of the IBC, and should be very reasonable to reject from a 1U chassis with fairly basic air cooling, especially since it's not all coming from a single point load like a single-socket server.

The code is at https://github.com/azonenberg/staticnet but I've intentionally avoided over-publicizing it since it hasn't had any kind of third party security review. As of now it's functional enough I'm willing to deploy it on a lab network but wouldn't trust it open to an untrusted network.

I work in embedded security and have tried to avoid any of the most gross footguns, deliberately simplifying the implementation as much as possible both to optimize for deeply embedded applications with double-digit kB flash/RAM footprint, and to minimize attack surface. The only supported cipher suite is ssh-ed25519 + aes128-gcm, and I didn't implement either of those (I use the DJB reference implementation or my line-by-line FPGA port of it for the 25519, and the hardware AES + RNG on the STM32).

But I've never done a formal code review myself (not that I'd trust one done by me as the author of the offending code, but it'd be a good starting point to find low-hanging fruit before I waste somebody else's time), fuzzed it, etc.

I have a bunch of additional features I want to add (notably, IPv6 support in the TCP/IP stack is very incomplete and not yet usable) and then am going to try to get at least some friends/coworkers to bang on it more.