Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

25 Gigabit Linux internet router PC build

michael.stapelberg.ch

Post reply on HN
Print view View original

281–290 of 359 posts

Re: 25 Gigabit Linux internet router PC build

#281

Post by eqvinox »

n3dm wrote: post #256
johnwalkr wrote: post #60

Earlier quoted context omitted.

I have 10G in Tokyo and I think it’s about $60 USD per month vs $40 for 2G. It’s definitely overkill but it sure is nice to download a 60GB steam game in a minute or two.

This hardly is relevant to your internet speeds with steam downloads. It is much more CPU dependent when you are operating at such high speeds.

A modern CPU + NIC combination will be twiddling its thumbs on a bulk download at 10G. There's a whole sleuth of optimization and offload features between the NIC and the CPU, and they work particularly well with a low number of high bandwidth data streams.

The real problem starts when you're doing lots of small packets all over the place. Which is not something you'll likely run into at home, ever.

Re: 25 Gigabit Linux internet router PC build

#282

Post by eqvinox »

deckard1 wrote: post #254
tyingq wrote: post #111

Earlier quoted context omitted.

I would go with a used SuperMicro A1SAI-2750F motherboard. It comes with an Atom 8-core C2750 (20w TDP) and a single 8 lane PCIe slot, but doesn't need active cooling. Supports ECC ram too. There's a passive heatsink on the CPU that works fine. I believe that would keep up with 2x10gb, be quiet, and power efficient. They are $200 or so used on eBay. So not terribly expensive for something that can push 10g. And it wi…

I'd probably go with SuperMicro X10SDV-4C-TLN2F. It already has 10GbE. Xeon D-1521 2x 10GbE ports onboard 6x SATA3 ports Accepts ECC RDIMM DDR4 (usually cheaper than UDIMM ECC) 1x PCIe 3.0 x16 1x M.2 slot There is one on ebay right now for $230 USD shipped. Also, if you're going for lower power consumption I believe you can disable the IPMI and the 2nd 10GbE port if you don't need it. There are other ways to reduce c…

The cheap ones (including the one you mentioned) are useless because they have 10Gbase-T. You need SFP+ slots to hook up your provider's fiber.

Which is kinda sad too because implementing 10Gbase-T is more expensive to implement and more expensive to run (because the 10Gb-T PHY burns power.) Sadly, there's not enough knowledge about this going around.

Re: 25 Gigabit Linux internet router PC build

#283

Post by eqvinox »

sydney6 wrote: post #252

To get a grip around the numbers, i quote from George Neville-Neil's Talk at BSDCan '15 "Measure Twice, Code Once" [1]: - 10 Gbps is 14.8 million 64 byte packets per second - 67.5 ns per packet or 200 cycles at 3 GHz - Cache miss is 32 ns [1] https://www.youtube.com/watch?v=LE4wMsP7zeA

You're not running a continuous stream of 64-byte packets in a home or SME setup. Also, assuming a 1:1 mapping to packet processing is a false dichotomy these days, NICs are doing an unbelievable amount of preprocessing, particularly grouping related packets together.

Re: 25 Gigabit Linux internet router PC build

#284

Post by coder543 »

InTheArena wrote: post #249
subhro wrote: post #203

Earlier quoted context omitted.

Odroid N2?

Need a lot more power then that to handle 2x 1GB/s connections.

What?

The UDM Pro[0] has a quad-core Cortex A57 CPU running at 1.7GHz, according to the spec sheet I'm looking at right now. It has 4GB of RAM.

The ODROID-N2+[1] has a quad-core Cortex A73 CPU running at 2.4GHz. It also has 4GB of RAM.

The UDM Pro is like half the performance of the ODROID-N2+, accounting for architecture and frequency differences, so... I'm not sure what you're getting at?

The ODROID should be more than enough to handle a gigabit connection if the UDM Pro is even halfway capable of it, from a power perspective. Connecting more ethernet ports to it without an exposed PCIe connector is going to be clunky, but that's not the issue you pointed out.

[0]: https://dl.ubnt.com/ds/udm-pro

[1]: https://www.hardkernel.com/shop/odroid-n2-with-4gbyte-ram-2/

Re: 25 Gigabit Linux internet router PC build

#285

Post by eqvinox »

InTheArena wrote: post #249
subhro wrote: post #203

Earlier quoted context omitted.

Odroid N2?

Need a lot more power then that to handle 2x 1GB/s connections.

No, you really don't. Though the problem in this case is the N2 only has 1 GbE port and adding more through USB, even 3.0, is notoriously bad (regardless of the platform.)

(source: I benchmarked my N2+. It'll route and NAT 1 GBit quite leisurely.)

Re: 25 Gigabit Linux internet router PC build

#286

Post by eqvinox »

tyingq wrote: post #111
sliken wrote: post #8

Was hoping for similar but 10 gbit. Anyone know a nice small/quiet linux box with 2x10 gbit for use as a router? I have a 8 port x 1gbit ubiquiti router, but ubiquiti seems to be going downhill, and 8 port x 10gbit routers are pretty expensive. My plan was a 2 port router and use vlan tagging to a 8+ port 10g switch. That way I can have separate networks for trusted ethernet, trusted wifi, untrusted ethernet, and unt…

I would go with a used SuperMicro A1SAI-2750F motherboard. It comes with an Atom 8-core C2750 (20w TDP) and a single 8 lane PCIe slot, but doesn't need active cooling. Supports ECC ram too. There's a passive heatsink on the CPU that works fine. I believe that would keep up with 2x10gb, be quiet, and power efficient. They are $200 or so used on eBay. So not terribly expensive for something that can push 10g. And it wi…

Note the Atom C2xxx series is the one where Intel f*cked up the LPC bus I/O drivers and it dies after a few years. Don't buy them used unless you're comfortable soldering a resistor to a clock line to extend its life.

This is fixed in the C0 stepping of the CPU, but finding out the CPU stepping is difficult even on new boards.

(cf. https://www.intel.com/content/dam/www/public/us/en/documents... - this is AVR54)

Re: 25 Gigabit Linux internet router PC build

#287

Post by RKearney »

nix23 wrote: post #250
RKearney wrote: post #240

Earlier quoted context omitted.

I'm sure the fact that Germany has a 566% larger population density and is 96.19% smaller than the United States has a lot to do with a countries ability to bring 25Gbps internet to its citizens.

>I'm sure the fact that Germany Sorry to disappoint you, but it not Sweden or Germany but Switzerland...Germany has bad internet too, but hey that mistake happens even to the best ones (like Bush Junior)

Not a disappointment at all. I see now that it is indeed Switzerland. The mention of the Germany based IX upgrading to 25Gbps towards the end of the article threw me off.

But Switzerland has a 504% larger population density and 99.56% smaller land area as compared to the United States so my point still stands.

Re: 25 Gigabit Linux internet router PC build

#288

Post by gunapologist99 »

Bud wrote: post #220

The real story here is how great the internet service can be now in a civilized country if you don't let Comcast have monopolies and ban local internet cooperatives and coast along selling the same service for 15 years while raising prices. 25Gbps symmetric for about $70/month. That's significantly less than I pay Comcast for its crappy 900Mbps down/40Mbps up service.

I feel the pain. I pay $80/mo for 200Mb/s down.

But, still, installing fiber is just expensive.

Much more expensive than almost anything else, and requires skill and experience to terminate the connections, and that's even overlooking any regulatory hurdles of digging along the right-of-way, private and public land ownership issues, the costs of the equipment, and the fragility of the fiber itself once installed.

None of those costs really scale. In fact, they increase linearly as the pool of available installers decreases, and that's not even talking about the cost of connecting that fiber to anything on the other end.

Re: 25 Gigabit Linux internet router PC build

#289

Post by sydney6 »

eqvinox wrote: post #283
sydney6 wrote: post #252

To get a grip around the numbers, i quote from George Neville-Neil's Talk at BSDCan '15 "Measure Twice, Code Once" [1]: - 10 Gbps is 14.8 million 64 byte packets per second - 67.5 ns per packet or 200 cycles at 3 GHz - Cache miss is 32 ns [1] https://www.youtube.com/watch?v=LE4wMsP7zeA

You're not running a continuous stream of 64-byte packets in a home or SME setup. Also, assuming a 1:1 mapping to packet processing is a false dichotomy these days, NICs are doing an unbelievable amount of preprocessing, particularly grouping related packets together.

No, of course not. A good starting point for real world performance benchmarking could be e.g. IMIX [1].

The example above represents the solely theroretical worst case as a means to establish a baseline for performance benchmarking.

Anyway, if you are referring to HW offloading capabilities of "modern" NIC's, using techniques like LRO would break the "end-to-end"-principle of a router.

[1] https://en.wikipedia.org/wiki/Internet_Mix

Re: 25 Gigabit Linux internet router PC build

#290

Post by cortesoft »

j1elo wrote: post #4

I'm always amazed by people who don't bat an eye on the perspective of having a home server, sucking up electricity 24/7. In this case replacing a typical router (which consumes like a lightbulb), with a full-fledged PC (probably consuming like x100 the power). I guess some people around the world have quite cheap utility bills! For me, it's either a Raspberry Pi type of power consumption, or else a server that only…

My solar panels on my roof generate about 10KW/h more than I use. I can bank credits with the grid, but can never cash them out, so electricity is basically free for me.
Post reply on HN