Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PCI Express on the Raspberry Pi 4

mloduchowski.com

Post reply on HN
Print view View original

141–150 of 166 posts

Re: PCI Express on the Raspberry Pi 4

#141

Post by morphle »

HeWhoLurksLate wrote: post #118
morphle wrote: post #111

Earlier quoted context omitted.

Both. The FPGA can interconnect the 16 RPi4s at 40 Gbps and also interconnect the 16 1G Ethernet at 16 Gbps, even interconnect the 16 HDMI, MPI and GPIO, depending on the FPGA. The FPGA can add 256 GB DDR3 and lots of other IO's like SATA and HDMI. (see my other comment for a $159 FPGA). The FPGA can act like a switch, an IO and memory extender and still have room for up to 300 ARM or Risc-V softcores.

What FPGA are you planning on using?

See my other posts on this page. You want FPGA's with cheap SERDES links that support PCIe Gen 2 at 5 Ghz. The best fit is Lattice ECP5-5G but that's $5 per link. The MPF300 is $10 per 12,5 Gbps link on the discounted developmentboard (with desoldering). A retail priced Cyclone 10CX105 also $10 per link with a smaller 10CX105 at $14.6. But very potent FPGAs that can be a small GPU in itself.

I now plan a crowdfunding for our own FPGA in an ASIC, that would bring $0.25 with a hundred links. This HN pages shows me there will be enough buyers for a $300 364 core RPI4 compatible cluster (100 BCM2711 chips connected to 1 FPGA plus 100GB DDR4) but without the RPi4 boards. Instead of attaching RPi4 or BCM2711, you could have 100 SATA, SSD, 30 HDMI, 10G or a rack full of PCIe servers connected to this interconnect FPGA. You are welcome to help realise the project or the crowdfunding.

Re: PCI Express on the Raspberry Pi 4

#142

Post by gnode »

kllrnohj wrote: post #137
gnode wrote: post #134

Earlier quoted context omitted.

Thunderbolt 3 controllers have a 4x link to provide one or two ports or 2x in the case of JHL6240. Additionally PCIe is designed to support backwards compatibility and link scaling. I don't see any reason why the 1x gen2 lane of the pi 4 couldn't host a Thunderbolt 3 port; it would just severely bottleneck the bandwidth of tunnelled PCIe links. Even though it would be limited, a Thunderbolt 3 port would expand the co…

Sure but "hey here's 8x thunderbolt 3 ports just don't ever attempt to use an entire one at once kthx" isn't exactly going to be a great product story, either. > I don't see any reason why the 1x gen2 lane of the pi 4 couldn't host a Thunderbolt 3 port; it would just severely bottleneck the bandwidth of tunnelled PCIe links. But that's kind of literally the reason? An entire ecosystem of products assumes a reasonably…

I agree that 8x Thunderbolt 3 is probably excessive, and I wouldn't want to trade away the current connectivity options as was suggested.

> If you take away the bandwidth from Thunderbolt 3 it just becomes USB

It becomes low bandwidth Thunderbolt / PCIe. You could still use it to attach PCIe devices which don't need a lot of bandwidth. GPUs can be attached for high performance compute where CPU-GPU bandwidth isn't critical. PCIe has non-bandwidth benefits over USB such as DMA and interrupts.

> why not just offer USB connectors which have even broader support and not as many cabling restrictions?

You can't attach PCIe devices via USB, but you can attach USB and PCIe devices via Thunderbolt.

Re: PCI Express on the Raspberry Pi 4

#144

Post by anderspitman »

noobermin wrote: post #32

Now this is the content I come to HN for. A serious hack just days after the 4 was released. Kudos to the OP. I envy people like OP for their tenacity. I barely have time to follow what's happening in IT, much less get ahead of the pack in doing cools hacks like this.

I'm sure you are but just in case are you aware of hackaday.com?

Re: PCI Express on the Raspberry Pi 4

#145

Post by hajile »

buildbuildbuild wrote: post #72

Fun. If you want PCIe on a SBC without the soldering, I highly recommend perusing Hackerboards. I'm very happy with my RockPi 4 (4GB RAM, PCIe, USB 3, 6 cores), which I discovered through their excellent database. https://www.hackerboards.com/search_boarddb.php

If I were moving away from the Raspberry Pi 4, I'd definitely consider the Nvidia Jetson Nano. It comes with a massive cooler attached. Pi 4 needs a cooler which will run you around $20. That puts you rather close to the nano in price ($75 vs $100), but the nano also has a GPU that is enormously more powerful and well worth the extra $25. Not needing special HDMI cables (or adapters) for the Nano is another money saver.

The biggest factor though is support. Raspberry Pi has a lot of software support, so you aren't running into weird bugs here and there with nobody around to help. The Jetson community isn't nearly as big, but Nvidia's track record on their software support is generally quite good. In this case, they have an extra interest given their push for commercial applications and that the X1 sees use in the Nintendo Switch and Shield TV (among other things).

Re: PCI Express on the Raspberry Pi 4

#146

Post by anderspitman »

morphle wrote: post #141
HeWhoLurksLate wrote: post #118

Earlier quoted context omitted.

What FPGA are you planning on using?

See my other posts on this page. You want FPGA's with cheap SERDES links that support PCIe Gen 2 at 5 Ghz. The best fit is Lattice ECP5-5G but that's $5 per link. The MPF300 is $10 per 12,5 Gbps link on the discounted developmentboard (with desoldering). A retail priced Cyclone 10CX105 also $10 per link with a smaller 10CX105 at $14.6. But very potent FPGAs that can be a small GPU in itself. I now plan a crowdfunding…

I have no idea what I'd use this for but it sounds awesome.

Re: PCI Express on the Raspberry Pi 4

#147

Post by morphle »

anderspitman wrote: post #146
morphle wrote: post #141

Earlier quoted context omitted.

See my other posts on this page. You want FPGA's with cheap SERDES links that support PCIe Gen 2 at 5 Ghz. The best fit is Lattice ECP5-5G but that's $5 per link. The MPF300 is $10 per 12,5 Gbps link on the discounted developmentboard (with desoldering). A retail priced Cyclone 10CX105 also $10 per link with a smaller 10CX105 at $14.6. But very potent FPGAs that can be a small GPU in itself. I now plan a crowdfunding…

I have no idea what I'd use this for but it sounds awesome.

Particle systems [1].

AR. VR [2][3][4][5].

Image processing, neural nets.

But not in programming languages like C, Java or Linux.

Only in massively parallel message passing languages[4].

I suggest a Smalltalk message passing scalable cluster VM like the RoarVM [6][7][]8].

[1] Shadama 3D. Yoshiki Oshima. https://www.youtube.com/watch?v=rka3w90Odo8

[2] OpenCobalt Alpha. https://www.youtube.com/watch?v=1s9ldlqhVkM&t=13s

[4] OpenCroquet Alan Kay Turing lecture. https://www.youtube.com/watch?v=aXC19T5sJ1U?t=3581

[5] OpenCroquet Alan Kay et al. https://www.youtube.com/watch?v=XZO7av2ZFB8

[6] RoarVM paper. David Ungar et al. https://stefan-marr.de/renaissance/ungar-adams-2009-hosting-...

[7] RoarVM Demo. https://www.youtube.com/watch?v=8pmVAtk612M

[8] RoarVM Demo. https://www.youtube.com/watch?v=GeDFcC6ps88

Re: PCI Express on the Raspberry Pi 4

#148

Post by herogreen »

hajile wrote: post #145
buildbuildbuild wrote: post #72

Fun. If you want PCIe on a SBC without the soldering, I highly recommend perusing Hackerboards. I'm very happy with my RockPi 4 (4GB RAM, PCIe, USB 3, 6 cores), which I discovered through their excellent database. https://www.hackerboards.com/search_boarddb.php

If I were moving away from the Raspberry Pi 4, I'd definitely consider the Nvidia Jetson Nano. It comes with a massive cooler attached. Pi 4 needs a cooler which will run you around $20. That puts you rather close to the nano in price ($75 vs $100), but the nano also has a GPU that is enormously more powerful and well worth the extra $25. Not needing special HDMI cables (or adapters) for the Nano is another money sav…

More like 10$ ? https://shop.pimoroni.com/products/fan-shim

Re: PCI Express on the Raspberry Pi 4

#149

Post by ComputerGuru »

sebazzz wrote: post #88
qdot_me wrote: post #14

Earlier quoted context omitted.

It’s all, cough, implementation dependent. You can run two VideoBIOS implementations, and chances are only one will install an int10h handler, but both will set up the registers and whatnot in the GPUs. My AMD TAHITIs for instance need VideoBIOS to start some form of thermal management loop - otherwise they just run full-throttle on the fan. Then whichever card prevails (BIOS has the ability to select the initializat…

I always assumed the VideoBIOS runs on the GPU, but if I understand you correctly this is code run on the CPU?

Yes, the PC (or whatever) hands off control of the entire system to the VideoBIOS which must do its thing and then pass control back.

Re: PCI Express on the Raspberry Pi 4

#150

Post by kllrnohj »

gnode wrote: post #142
kllrnohj wrote: post #137

Earlier quoted context omitted.

Sure but "hey here's 8x thunderbolt 3 ports just don't ever attempt to use an entire one at once kthx" isn't exactly going to be a great product story, either. > I don't see any reason why the 1x gen2 lane of the pi 4 couldn't host a Thunderbolt 3 port; it would just severely bottleneck the bandwidth of tunnelled PCIe links. But that's kind of literally the reason? An entire ecosystem of products assumes a reasonably…

I agree that 8x Thunderbolt 3 is probably excessive, and I wouldn't want to trade away the current connectivity options as was suggested. > If you take away the bandwidth from Thunderbolt 3 it just becomes USB It becomes low bandwidth Thunderbolt / PCIe. You could still use it to attach PCIe devices which don't need a lot of bandwidth. GPUs can be attached for high performance compute where CPU-GPU bandwidth isn't cr…

You could also do all that with just a PCI-E x1 slot and use the x1 to remote x16 connector referenced in the blog post to extend it. No reason to mess with Thunderbolt just to have any PCIE capability at all.
Post reply on HN