Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PCI Express on the Raspberry Pi 4

mloduchowski.com

Post reply on HN
Print view View original

111–120 of 166 posts

Re: PCI Express on the Raspberry Pi 4

#111

Post by morphle »

rowanG077 wrote: post #47
qdot_me wrote: post #9

Earlier quoted context omitted.

There are two ways of doing clusters - one is a message passing paradigm, which you can do over Ethernet (to an extent - I’d still take USB3 for 4x the bandwidth) - and the other is direct memory access a’la Cray. What really motivated me to do this hack is the relative abundance of stuff I can now plug into an FPGA :)

So your plan is to attach an FPGA to the PCIe bus? To allow the FPGA to access peripherals on the Pi side or do you want the FPGA to make the pi a lot more powerful?

Both. The FPGA can interconnect the 16 RPi4s at 40 Gbps and also interconnect the 16 1G Ethernet at 16 Gbps, even interconnect the 16 HDMI, MPI and GPIO, depending on the FPGA. The FPGA can add 256 GB DDR3 and lots of other IO's like SATA and HDMI. (see my other comment for a $159 FPGA). The FPGA can act like a switch, an IO and memory extender and still have room for up to 300 ARM or Risc-V softcores.

Re: PCI Express on the Raspberry Pi 4

#112

Post by beatgammit »

nereid wrote: post #97

Interesting install a SATA card and make a NAS. I think better than a usb.

I would love something with 4x SATA ports for a NAS like this one[1]. I've seen PCIe on devices like these, but I've heard there are issues getting drivers to work properly. I haven't actually tried it, but other limitations (RAM, CPU, Ethernet) have prevented me from actually giving it a shot (I want ZFS, which is a bit memory hungry). The Pi has just enough that I think it's doable.

I would absolutely love it if the Raspberry Pi foundation made a version with PCIe instead of USB.

- [1] https://www.newegg.com/syba-si-pex40064-sata-iii/p/N82E16816...

Re: PCI Express on the Raspberry Pi 4

#113

Post by gambiting »

zokier wrote: post #107
Millennium wrote: post #101

If a Pi is capable of this already, why not replace the Ethernet, charging, micro-HDMI, and USB ports with a boatload of type-C Thunderbolt ports (plus support for the HDMI 1.4 alt mode)? Would 8xUSB-C cost that much more than 1xUSB-C+1xEthernet+2xMicro-HDMI+2xUSB3+2xUSB2 (with no PCI Express), in exchange for a considerably more flexible device?

Because people do not want to deal with gazillion dongles

And not just because all those dongles would significantly increase the cost of a relatively cheap device.

Re: PCI Express on the Raspberry Pi 4

#114

Post by [deleted] »

Millennium wrote: post #101

If a Pi is capable of this already, why not replace the Ethernet, charging, micro-HDMI, and USB ports with a boatload of type-C Thunderbolt ports (plus support for the HDMI 1.4 alt mode)? Would 8xUSB-C cost that much more than 1xUSB-C+1xEthernet+2xMicro-HDMI+2xUSB3+2xUSB2 (with no PCI Express), in exchange for a considerably more flexible device?

[deleted]

Re: PCI Express on the Raspberry Pi 4

#115

Post by Millennium »

joey_bob wrote: post #108
Millennium wrote: post #101

If a Pi is capable of this already, why not replace the Ethernet, charging, micro-HDMI, and USB ports with a boatload of type-C Thunderbolt ports (plus support for the HDMI 1.4 alt mode)? Would 8xUSB-C cost that much more than 1xUSB-C+1xEthernet+2xMicro-HDMI+2xUSB3+2xUSB2 (with no PCI Express), in exchange for a considerably more flexible device?

A lot of the benefit of the rpi is the built in IO. Would the rpi still be cost effective without it? Not for many of the IoT projects that make the rpi so popular.

I didn't propose replacing the GPIO, or the camera port, or the other display port. I'm talking specifically about the other ports around the sides of the device, which are already used for more traditional computer-y things. Of course the Pi needs to retain its built-in low-level I/O, but it has another side to it too, and that's the side I'm talking about using Thunderbolt with.

Re: PCI Express on the Raspberry Pi 4

#116

Post by cr0sh »

whitelabrat wrote: post #66
jamesfmilne wrote: post #52

Earlier quoted context omitted.

NVIDIA drivers might even work too Linux 32-bit ARM drivers https://www.nvidia.co.uk/Download/driverResults.aspx/137468/...

This would be incredible if gotten to work with a pi. I've been eyeing the Jetson Nano, but if I could use a pi with a V100, that would be hilarious. And awesome.

Based on what I've seen - it's on my list of projects to "try out", but rapidly becoming a more obsolete idea the longer I wait (but then again, I've got an Altair on my todo list, so meh)...

Taking a low-cost Mini ITX board and pairing it with a decent mini-sized GTX 750 TI SC video card (I prefer the EVGA one) should yield a system that is somewhat on par with some of NVidia's embedded ML offerings, with the tradeoff of size and efficiency - but at a significantly reduced cost.

Add a very short PCIe riser ribbon (or, if you can find one tall enough, a right-angle riser), and you could lay the card over "flat" above the CPU (using a 1U cooler/heatsink); you'd have to make a custom mounting frame of course, but I think you could make the whole package relatively compact.

From my limited experience using Tensorflow with the 750 - it is a very capable card in that capacity and relatively inexpensive today. If you were willing to spend a bit more, there are mini-sized NVidia GPUs available in more recent models; of course if space is not an issue, then full-length GPUs can be substituted (my "goal" was to build such a system as close to within the footprint of a Mini-ITX board as possible).

As far as the Mini-ITX motherboard is concerned, again, size to your budget. That said, if you decided to build this system using cheap pre-owned components, maybe an I5 with 8 gig - you could probably get it going for under $250.00 USD, maybe less.

Re: PCI Express on the Raspberry Pi 4

#117

Post by _ea1k »

beatgammit wrote: post #98
buildbuildbuild wrote: post #72

Fun. If you want PCIe on a SBC without the soldering, I highly recommend perusing Hackerboards. I'm very happy with my RockPi 4 (4GB RAM, PCIe, USB 3, 6 cores), which I discovered through their excellent database. https://www.hackerboards.com/search_boarddb.php

I think you mean the RockPro64. The Rock64 only has 4 cores and no PCIe. That being said, I missed the PCIe on the specs last time I was comparing SOCs and I had forgotten about hackerboards, thanks for the reminder!

Amusingly, there appears to be no relationship between the "Rock Pi" and the Rock64/RockPro64 family. These names are really confusing.

Re: PCI Express on the Raspberry Pi 4

#118

Post by HeWhoLurksLate »

morphle wrote: post #111
rowanG077 wrote: post #47

Earlier quoted context omitted.

So your plan is to attach an FPGA to the PCIe bus? To allow the FPGA to access peripherals on the Pi side or do you want the FPGA to make the pi a lot more powerful?

Both. The FPGA can interconnect the 16 RPi4s at 40 Gbps and also interconnect the 16 1G Ethernet at 16 Gbps, even interconnect the 16 HDMI, MPI and GPIO, depending on the FPGA. The FPGA can add 256 GB DDR3 and lots of other IO's like SATA and HDMI. (see my other comment for a $159 FPGA). The FPGA can act like a switch, an IO and memory extender and still have room for up to 300 ARM or Risc-V softcores.

What FPGA are you planning on using?

Re: PCI Express on the Raspberry Pi 4

#119

Post by qdot_me »

bstar77 wrote: post #83
qdot_me wrote: post #6

Earlier quoted context omitted.

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itaniums in the past. VideoBIOS expects to run and expects a well behaving Intel CPU to do the power-up. That said X can sometimes emulate these quite well. On ARM we’d also run into alignment issues and likely other quirks - but in principle...

"GPUs are notoriously hard to get to work on non-intel" I don't understand this statement... accelerated graphics have no issues working on any x86 system as well as ARM based linux systems (not to mention every cell phone in the past 10 years). Itanium is also Intel so that statement contradicts your previous statement.

To a limited extent. The accelerated graphics on ARM (and cell phones) has always been a system integrator problem. Until very recently, they were not user customizable - which means the initialisation was part of the proprietary firmware and hard to port across even different implementations of the same SoC.

The x86 world had the advantage of user upgradable GPUs, which necessitates standardization and common firmware.

On top of which, the accelerated graphics of cell phones is a horrible kludge of various standards.

Re: PCI Express on the Raspberry Pi 4

#120

Post by monocasa »

benj111 wrote: post #91
monocasa wrote: post #85

Earlier quoted context omitted.

I've seen pcie literally run over a metal clothes hanger soldered to the board. It's extremely tolerant of terrible quality connections.

How did you reach the situation where the solution was soldering a clothes hanger to a board for pcie ???

We were seeing some issues, one engineer was blaming signal integrity issues, we didn't have access to a high enough speed oscope to get a clean eye diagram, and so another engineer literally disappeared for an hour and had the boards running over the clothes hangars on the old firmware to say, no, it's not a questionable signal integrity issue, go back and fix your code.
Post reply on HN