Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building an ARM64 home server the hard way

jforberg.se

Post reply on HN
Print view View original

71–80 of 119 posts

Re: Building an ARM64 home server the hard way

#71

Post by throwaway173738 »

jforberg wrote: post #8
daneel_w wrote: post #2

For the Pine family of SBCs I highly recommend installing Tow-Boot - https://tow-boot.org/ - on the SPI flash memory to allow yourself much better boot options, including booting directly from NVMe so you don't need to keep the MicroSD card plugged-in.

Yes, I considered that and agree that it would have been nicer! I didn't pursue it for this project because my jury-rigged SD boot was working fine and I wanted to move on to other parts of the system.

SD cards are pretty unreliable so it might be good to take it out of the loop at some point. Most field failures I’ve seen for products I develop have been related to SD cards dying during power loss or falling out due to vibration.

Re: Building an ARM64 home server the hard way

#72

Post by Tepix »

tbrock wrote: post #23

You could do this for cheaper (and with more ram) using a raspberry pi 4 with a usb nvme ssd, it’s got gigabit ethernet and is arm64. Sure you have two less cores than this solution but it’s more likely to be supported over time and once you get the SD card out of the mix the I/O is solid. I’ve been surprised by how much the SD card throughput was limiting the experience. I run arch Linux arm on mine and it’s a fanta…

Any SSD will do really since it'll be bottlenecked by 5GBit/s USB.

Re: Building an ARM64 home server the hard way

#73

Post by jamesy0ung »

megous wrote: post #66
tbrock wrote: post #23

You could do this for cheaper (and with more ram) using a raspberry pi 4 with a usb nvme ssd, it’s got gigabit ethernet and is arm64. Sure you have two less cores than this solution but it’s more likely to be supported over time and once you get the SD card out of the mix the I/O is solid. I’ve been surprised by how much the SD card throughput was limiting the experience. I run arch Linux arm on mine and it’s a fanta…

RK3399 is completely FOSS, from bootloader, to firmware, to Linux drivers and mainlined. It will be supported as long as someone wants to run software on it.

Raspberry Pi, despite not being fully FOSS, has a much better community supporting it. I’d bet it will last longer than the pine boards. Watched Jeff Geerling’s videos on it and he had trouble getting the pine board to work, whereas the Raspberry Pi worked first try. The only images for it were some images a guy made and put on the pine wiki.

Re: Building an ARM64 home server the hard way

#74

Post by MobiusHorizons »

I have this same board running freebsd as a NAS. It has been a pretty great experience overall. My main gripe has been that I would really like to be able to run an nvme drive for a cache at the same time as SATA ports, and I haven't found any cost effective PCIE2.0 x4 switches that I could use for that purpose. There is an x1 switch for use with RP4, but it is a shame to lose all the bandwidth.

I'm looking forward to someone making a NAS board on the new RK3588 since that has enough connectivity for everything I want.

Re: Building an ARM64 home server the hard way

#75

Post by daneel_w »

throwaway173738 wrote: post #70
daneel_w wrote: post #2

For the Pine family of SBCs I highly recommend installing Tow-Boot - https://tow-boot.org/ - on the SPI flash memory to allow yourself much better boot options, including booting directly from NVMe so you don't need to keep the MicroSD card plugged-in.

Does the U-boot in the SPI-NOR not support booting from NVMe? It might also be possible to patch that in from mainline if it exists. You can also often provide a “boot script” in the vfat partition that overrides the boot config in non volatile memory. This was something Freescale did with the i.MX6 that became a relatively standard thing for vendor-supplied U-boot.

The default setup does not support it for any Rockchip SoC up to and including the RK3399, which is why everyone goes for tow-boot on SPI for their Pine devices. The SoCs have a hardcoded boot sequence which includes SPI, SD, USB and eMMC, but not PCIe. It is however available since the RK3588 - e.g. the Radxa Rock5B boots from NVMe right off the bat.

Re: Building an ARM64 home server the hard way

#77

Post by oynqr »

jamesy0ung wrote: post #73
megous wrote: post #66

Earlier quoted context omitted.

RK3399 is completely FOSS, from bootloader, to firmware, to Linux drivers and mainlined. It will be supported as long as someone wants to run software on it.

Raspberry Pi, despite not being fully FOSS, has a much better community supporting it. I’d bet it will last longer than the pine boards. Watched Jeff Geerling’s videos on it and he had trouble getting the pine board to work, whereas the Raspberry Pi worked first try. The only images for it were some images a guy made and put on the pine wiki.

That's the great thing about mainline support, you ignore random trashy images and use generic distro ones.

Re: Building an ARM64 home server the hard way

#78

Post by malteg »

candiddevmike wrote: post #56
tyingq wrote: post #46

Earlier quoted context omitted.

Not cheap, but does support ECC: https://morefine.com/products/morefine-s500-mini-pc The FAQ question reads a bit funny: "Q19:Support ECC RAM？ YES. S500+ Support ECC RAM， But compatibility requirements are higher, and cannot be used casually." Also, these AsRock products: https://www.asrockind.com/en-gb/4X4%20BOX-V1000M https://www.asrockind.com/en-gb/4X4%20BOX-R1000M

Thank you for the links, that morefine one seems really nice, would need to get 2.5Gb networking gear... I currently have a single 16 core/64 GB ECC Ryzen tower with ZFS that I'd like to break into a ZFS ARM ECC NFS server with a separate small form factor Ryzen or ARM compute cluster.

Honeycomb? https://www.solid-run.com/arm-servers-networking-platforms/h... I think they also have a lite version supporting ECC…

Re: Building an ARM64 home server the hard way

#79

Post by drdaeman »

This seems to be low-power entry-level stuff. I'm curious, is there anything more serious - but less serious than some proper rack server hardware?

Currently I'm running a home server on EPYC 3251 mini-ITX board, which I use to route 1GbE WAN and 10GbE LAN, serve as a NAS, and run a bunch of services all without it breaking a sweat, and leaving plenty of headroom shall I want to run more stuff there. It sits on my desk in a small-ish cubic Supermicro chassis and barely makes any noise beyond the normal HDD screeching. And it's an entry-level server-oriented board so I have proper LOM without having to throw in an IPKVM.

I would fancy an ARMv8 machine - just for fun of it (and possibly better performance per watt) - but I think I can't get anything comparable from a RPi-level hardware. But the next "step" I see when searching for ARM servers are those fan-screaming behemoths you put in a rack in a proper server room, which is something I dread for a homelab, as I don't have a dedicated room for it. I've had a pleasure of WfH involving setting up some PowerEdges in my living room, was fun but extremely noisy. So I wonder, where are the middle grounds?

Re: Building an ARM64 home server the hard way

#80

Post by lostlogin »

radiator wrote: post #26
doublepg23 wrote: post #12

Earlier quoted context omitted.

This is the way. An x86 NUC/Mini PC/Thin client smokes the ARM SBC market.

Are there any boxes without a graphics card or at least with a primitive one? So that they would be more economical in case you only need them for the role of a server?

All the small Nucs just use an iGPU. Boxes like the Nuc 8 have a bit of a cult following as the iGPU is surprisingly powerful. It’s probably beaten by newer models now. The newer ones have lots of cores, and when fully loaded with memory make a handy little box for VMs.
Post reply on HN