Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introducing “6-pack”: the first open hardware modular switch

code.facebook.com

Post reply on HN
Print view View original

61–70 of 97 posts

Re: Introducing “6-pack”: the first open hardware modular switch

#61

Post by korethr »

ay wrote: post #25
korethr wrote: post #19

This is rather cool, and I'm glad to see some work being done in the networking gear space utilizing open designs and firmware like this. Unfortunately, unless I find myself needing to network a datacenter in the near future, it's not immediately useful to me. That said, I do hope other developments in network gear that will be useful in other markets emerge from this effort. From my perspective, there's a gap in net…

I've the exact same needs, and by occasion got hold of cisco SG300-10 switch, which I think fits the description - so I thought to share my experience, in case it is useful. I use it for the home lab, where I have a one-armed router serving multiple VLANs, and I have 3x MacMinis running Linux as a "server farm" (The latter I use because they are quite a good gear power-management wise, scaling from ~18W at idle up to…

The specs on this look very nice indeed. I'll be taking a closer look at this.

My main hesitation is the proprietary firmware. Now I'm not going to disagree with anyone arguing that Cisco knows what they're doing and is competent at putting together firmware for the hardware they sell. Nor will I disagree with anyone arguing that open source is not a magical talisman ensuring quality. No, my main concerns are a) timeliness of critical updates, and useful lifetime of the hardware vs support lifetime.

Being open source cannot prevent bugs, but once found, fixes tend to become available quickly. Also, I have found that the useful lifetime of computing and networking hardware tends to exceed the period of time the vendor will offer support for it. I have gigabit ethernet switches I bought years ago that still work just fine, even though they're no longer sold. That's what I love about OpenWRT. The hardware my home router uses is discontinued, but still does the job just fine, and I can still get updates when I need to.

That said, I think I will be checking the SG300 out. Thanks for the recommendation.

Re: Introducing “6-pack”: the first open hardware modular switch

#62

Post by epistasis »

e12e wrote: post #49
virtuallynathan wrote: post #41

Earlier quoted context omitted.

All of these things are based on the Broadcom Trident II chip. If you want it cheap, don't go to juniper - get one from Quanta. You can have a 32x40G switch for $6,000.

On a similar note, I just recently became aware of Cumulus[1] Debian based switches (but the good bits are closed source) from among others edge-core[2] (via a presentation by PaaS-provider http://zetta.io ) -- eg: http://whiteboxswitch.com/products/edge-core-as4600-54t [1] http://cumulusnetworks.com/support/faq/ [2] http://www.edge-core.com/

We've been loving Cumulus + Quanta for 10Gb and 40Gb, in that it's more manageable than Cisco (for our environment) and a fraction of the cost. We end up using it at 1Gb too, but it's just a price match there, instead of a win.

Re: Introducing “6-pack”: the first open hardware modular switch

#63

Post by btoptical »

These big switch boxes typically end up being about thermal management and this box looks like thermal design was an afterthought. Also I'm not sure the power entry design is really all that smart. Generally speaking if you're in the market for a 640Gb/s switch (or 3.8Tb/s switch), does your data center really not have access to 48V power? The AC/DC conversion wastes power and space.

The QSFP's are "spaced for optimal airflow." However this spacing seems to neglect cooling the QSFPs themselves. Belly-to-belly mounting of modules is usually the most thermally challenging way to arrange them. The heat dissipated by the QSFP's is generally directed towards the top of the module. By placing open air channels between modules, they have effectively ensured that little to no air flows over the QSFP heat sinks (which is not shown). So there is probably a limitation on which reach codes are supported. My guess is that because of the thermal limitations of this design, it's not truly non-blocking in all reach configurations.

Re: Introducing “6-pack”: the first open hardware modular switch

#64

Post by wmf »

btoptical wrote: post #63

These big switch boxes typically end up being about thermal management and this box looks like thermal design was an afterthought. Also I'm not sure the power entry design is really all that smart. Generally speaking if you're in the market for a 640Gb/s switch (or 3.8Tb/s switch), does your data center really not have access to 48V power? The AC/DC conversion wastes power and space. The QSFP's are "spaced for optima…

does your data center really not have access to 48V power?

No, they really don't. I've never heard of 48V outside of telco.

Re: Introducing “6-pack”: the first open hardware modular switch

#66

Post by nl »

e12e wrote: post #55
gh02t wrote: post #23

Earlier quoted context omitted.

I also wanted something in the category you're describing and after shopping around a bit I stumbled across Mikrotik's products. I bought one of their 24 port smart switches and it was exactly what I was looking for. Not a lot more expensive than some of the nicer consumer-targeted gear, but it has way more functionality. http://routerboard.com/CRS125-24G-1S-IN

Looks like a very nice product (along with most of the other products). It's a little disappointing that they seem to do only the very minimum wrt the GPL: http://www.mikrotik.com/downloadterms.html I mean, sure, asking for 45 USD for a CD with the source is technically complying with the GPL -- but it does seem a bit strange in this day and age. Not to mention that for the source code to be useful, one would hope on…

Lots of PFSense people seem to think the PC Engines APU[1] is ok. Netgate sells a DIY kit[2].

It's only a 3 port system though.

[1] http://www.pcengines.ch/apu.htm

[2] http://store.netgate.com/kit-APU1C4.aspx

Re: Introducing “6-pack”: the first open hardware modular switch

#67

Post by kijiki »

e12e wrote: post #49
virtuallynathan wrote: post #41

Earlier quoted context omitted.

All of these things are based on the Broadcom Trident II chip. If you want it cheap, don't go to juniper - get one from Quanta. You can have a 32x40G switch for $6,000.

On a similar note, I just recently became aware of Cumulus[1] Debian based switches (but the good bits are closed source) from among others edge-core[2] (via a presentation by PaaS-provider http://zetta.io ) -- eg: http://whiteboxswitch.com/products/edge-core-as4600-54t [1] http://cumulusnetworks.com/support/faq/ [2] http://www.edge-core.com/

To clarify, the only part that is closed is "switchd", which is a userspace program that watches the kernel data structures (route tables, neighbor tables, bridges, ports, vlans, etc) and programs the hardware to match. It links against proprietary silicon vendor SDKs, and programs registers whose description were given to us under NDA.

Without this part, everything works the same, but is of course not hardware accelerated. So the 100% open source parts of Cumulus Linux would still make a great Network OS for a router/switch VM.

We don't yet have an official VM version, but that is something we will have in the future.

- nolan co-founder/CTO Cumulus Networks

Re: Introducing “6-pack”: the first open hardware modular switch

#68

Post by kijiki »

rsync wrote: post #3

I very much want to transition to open hardware models for networking gear - for both my own home use and for rsync.net. However, at no single site (especially my home) do I need a 6U chassis full of switch ports. Is there a 1U version of this on the horizon ?

Basically anything on here: http://cumulusnetworks.com/support/linux-hardware-compatibil...

Some easy ways to buy are: https://bm-switch.com/ http://whiteboxswitch.com/ http://ipfabrics.net/ http://www.penguincomputing.com/products/network-switches/

- nolan co-founder/CTO Cumulus

Re: Introducing “6-pack”: the first open hardware modular switch

#69

Post by btoptical »

wmf wrote: post #64
btoptical wrote: post #63

These big switch boxes typically end up being about thermal management and this box looks like thermal design was an afterthought. Also I'm not sure the power entry design is really all that smart. Generally speaking if you're in the market for a 640Gb/s switch (or 3.8Tb/s switch), does your data center really not have access to 48V power? The AC/DC conversion wastes power and space. The QSFP's are "spaced for optima…

does your data center really not have access to 48V power? No, they really don't. I've never heard of 48V outside of telco.

I've been to lots of them all over the world, names you'd recognize, and they've all had 48V.

Re: Introducing “6-pack”: the first open hardware modular switch

#70

Post by DiabloD3 »

e12e wrote: post #56
wmf wrote: post #54

Earlier quoted context omitted.

40GbE uses approximately the same copper cables as Infiniband. There's probably more used IB equipment floating around than used 40GbE, but otherwise I'd go with Ethernet.

Ah, of course. I was misled by wikipedia[1], but it should've been obvious that the same connection could be used for both inifinband and ethernet. So, indeed, there are copper interconnects: http://www.amazon.co.uk/Mellanox-3-3ft-QSFP-QSFP-Passive-Cop... Actually seems the price is finally coming down a bit (compared to what I remember these used to cost, years ago -- but maybe I've just upped my budget ;-). [1] htt…

The cable you're looking for is called a direct connect cable. "Fake" optical modules on both sides, plugs it together with a permanently wired copper cable.
Post reply on HN