Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PCI Express on the Raspberry Pi 4

mloduchowski.com

Post reply on HN
Print view View original

11–20 of 166 posts

Re: PCI Express on the Raspberry Pi 4

#11

Post by qdot_me »

segfaultbuserr wrote: post #5

It looks like an unreliable modification. Running a GHz-level interface with jumpers is almost impossible to control the impedance, it's a cool Proof-of-Concept though. But is it possible to bring the project to the next level? Is it possible to make a daughterboard with QFN connector? If so, one can make a pin-compatible daughterboard with an extension connector. To use it, just desolder the USB chip and solder a ne…

Yup. The daughterboard is on my mind. Likely flex-PCB and that’s gonna take a week or two to respin. Hence I’m collecting ideas for various daughtercards I could cram in a panel before sending it off - straight-through to riser via USB3, expresscard SMT, maybe through-hole 1x?

That said, PCIe phy’s are extremely robust - they do most of the impedance matching and delay mismatch training. And if you don’t ruin the onboard caps, this could be jumpered straight across.

Re: PCI Express on the Raspberry Pi 4

#12

Post by jchw »

qdot_me wrote: post #6
Jonnax wrote: post #2

That's really cool. I'm curious, would it be possible to use a modern GPU (running at 1x) on an ARM based board? Would the open source drivers that are part of the Kernel work out the box on ARM?

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itaniums in the past. VideoBIOS expects to run and expects a well behaving Intel CPU to do the power-up. That said X can sometimes emulate these quite well. On ARM we’d also run into alignment issues and likely other quirks - but in principle...

I always wondered what the deal was with Video BIOS on modern PCI-express graphics cards. Obviously it must exist, since int 10h still works.

Does video BIOS even need to be executed, though? I always assumed it was only necessary for the primary card to be able to display output during boot on PCs. (Otherwise, wouldn't two different cards trample over eachother's implementation of int 10h?)

Sorry for the probably-obvious questions, it's sometimes tricky to find good sources for weird information like this that are also up to date.

Re: PCI Express on the Raspberry Pi 4

#13

Post by pm215 »

qdot_me wrote: post #6
Jonnax wrote: post #2

That's really cool. I'm curious, would it be possible to use a modern GPU (running at 1x) on an ARM based board? Would the open source drivers that are part of the Kernel work out the box on ARM?

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itaniums in the past. VideoBIOS expects to run and expects a well behaving Intel CPU to do the power-up. That said X can sometimes emulate these quite well. On ARM we’d also run into alignment issues and likely other quirks - but in principle...

People have got at least some cards working on server-class arm setups. On the other hand I think they might be using the 'emulate x86 uefi rom in an arm uefi system' hack, so there might be more effort required on a non-uefi setup. Still, it should have shaken out most of the alignment and memory-cacheability-attibutes issues already, hopefully.

Re: PCI Express on the Raspberry Pi 4

#14

Post by qdot_me »

jchw wrote: post #12
qdot_me wrote: post #6

Earlier quoted context omitted.

Hack’s creator here - it’s on my list of things to try. GPUs are notoriously hard to get to work on non-intel, having tried to get a few up on Alpha and Itaniums in the past. VideoBIOS expects to run and expects a well behaving Intel CPU to do the power-up. That said X can sometimes emulate these quite well. On ARM we’d also run into alignment issues and likely other quirks - but in principle...

I always wondered what the deal was with Video BIOS on modern PCI-express graphics cards. Obviously it must exist, since int 10h still works. Does video BIOS even need to be executed, though? I always assumed it was only necessary for the primary card to be able to display output during boot on PCs . (Otherwise, wouldn't two different cards trample over eachother's implementation of int 10h?) Sorry for the probably-o…

It’s all, cough, implementation dependent. You can run two VideoBIOS implementations, and chances are only one will install an int10h handler, but both will set up the registers and whatnot in the GPUs.

My AMD TAHITIs for instance need VideoBIOS to start some form of thermal management loop - otherwise they just run full-throttle on the fan.

Then whichever card prevails (BIOS has the ability to select the initialization order) becomes the boot display device.

X11 has some (generally working, for well behaved GPUs) emulation of this environment, so that the GPU can initialize late, and even reset under X control. This is how sane cards can work under headless ARM etc.

Now, some manufacturers assume you get something like SSE or MMX - VideoBIOS spec technically mandates 386 instruction set only. That crap gets badly emulated.

On top of this, drivers can sometimes reinit anyways, from native kernel code. If that happens, the VideoBIOS concerns are moot.

Re: PCI Express on the Raspberry Pi 4

#15

Post by segfaultbuserr »

qdot_me wrote: post #11
segfaultbuserr wrote: post #5

It looks like an unreliable modification. Running a GHz-level interface with jumpers is almost impossible to control the impedance, it's a cool Proof-of-Concept though. But is it possible to bring the project to the next level? Is it possible to make a daughterboard with QFN connector? If so, one can make a pin-compatible daughterboard with an extension connector. To use it, just desolder the USB chip and solder a ne…

Yup. The daughterboard is on my mind. Likely flex-PCB and that’s gonna take a week or two to respin. Hence I’m collecting ideas for various daughtercards I could cram in a panel before sending it off - straight-through to riser via USB3, expresscard SMT, maybe through-hole 1x? That said, PCIe phy’s are extremely robust - they do most of the impedance matching and delay mismatch training. And if you don’t ruin the onb…

Great!

I was thinking about using a SMT ribbon cable connector because of the limited available space, but apparently it won't be an issue if you raise the board high enough?

Anyway, if this project ever goes to batch-production, make sure to update your blog when the funding campaign starts!

Re: PCI Express on the Raspberry Pi 4

#17

Post by jchw »

qdot_me wrote: post #14
jchw wrote: post #12

Earlier quoted context omitted.

I always wondered what the deal was with Video BIOS on modern PCI-express graphics cards. Obviously it must exist, since int 10h still works. Does video BIOS even need to be executed, though? I always assumed it was only necessary for the primary card to be able to display output during boot on PCs . (Otherwise, wouldn't two different cards trample over eachother's implementation of int 10h?) Sorry for the probably-o…

It’s all, cough, implementation dependent. You can run two VideoBIOS implementations, and chances are only one will install an int10h handler, but both will set up the registers and whatnot in the GPUs. My AMD TAHITIs for instance need VideoBIOS to start some form of thermal management loop - otherwise they just run full-throttle on the fan. Then whichever card prevails (BIOS has the ability to select the initializat…

Thanks for the information. I’ve been really curious about this with the incoming ARM dev boards that have full PCI-e edge connectors.

Pretty fascinating that X11 of all things is dealing with Video BIOS initialization, though. The sheer amount of functionality that was overloaded into X...

Re: PCI Express on the Raspberry Pi 4

#19

Post by hvidgaard »

qdot_me wrote: post #9
hvidgaard wrote: post #7

Earlier quoted context omitted.

Wouldn't the gigabit LAN be a better fit for this? If you want to make a cluster, you need to make some custom hardware to connect to, that can facilitate the communication. At this point you're likely spending more that if you just bought a real desktop for more performance. I can see the fun factor in hacking the system together, though.

There are two ways of doing clusters - one is a message passing paradigm, which you can do over Ethernet (to an extent - I’d still take USB3 for 4x the bandwidth) - and the other is direct memory access a’la Cray. What really motivated me to do this hack is the relative abundance of stuff I can now plug into an FPGA :)

I was thinking Ethernet because, A:It's cheap to buy a switch and cluster 100 RPi, B: You can have a desktop with a faster NIC and keep the RPis busy.

But as with everything high performance, it depends entirely on the use case.

Re: PCI Express on the Raspberry Pi 4

#20

Post by qdot_me »

hvidgaard wrote: post #19
qdot_me wrote: post #9

Earlier quoted context omitted.

There are two ways of doing clusters - one is a message passing paradigm, which you can do over Ethernet (to an extent - I’d still take USB3 for 4x the bandwidth) - and the other is direct memory access a’la Cray. What really motivated me to do this hack is the relative abundance of stuff I can now plug into an FPGA :)

I was thinking Ethernet because, A:It's cheap to buy a switch and cluster 100 RPi, B: You can have a desktop with a faster NIC and keep the RPis busy. But as with everything high performance, it depends entirely on the use case.

True. And with RPi4 having a 1000baseT, it’s not as painful as it seems. Perhaps even the driver can be coaxed into some form of DMA and MPI that is a bit lower latency than IP stack.

With secondary IP layer on 802.11, it might actually work reasonably well.

Post reply on HN