Live data from Hacker News

Unscrewed: A Story About OpenBSD

skeptech.org

21–30 of 67 posts

Re: Unscrewed: A Story About OpenBSD

#21
I'll never forget this time in college when I had set up an OpenBSD router in our apartment and came home one weekend to find that I was unable to log into it via SSH.

Actually, it was timing out.

So I hooked up a monitor and a keyboard to this old old P1/133 with 32MB of ram that had been taken out of a dumpster to be our router to see what was going on.

A few months prior to this I had been playing around with rrd and started generating traffic graphs so that I could get pretty pretty pictures of our traffic (and to figure out when one of my roommates was streaming his porn).

I had set the cron jobs to run every 10 minutes. Initially the data sets were so small that this wasn't an issue, but over time the database got larger and suddenly the cron job was taking 10 minutes and 10 seconds, and then 10 minutes and 20 seconds... on and on until finally cron jobs were running over cron jobs.

So after about 2 minutes waiting to get logged in, I finally get the load average for the system and it was somewhere around 146.20. After another couple of minutes, I was able to disable the cron jobs and kill off rrd graphs. In another 10 minutes, the load average was back to All this time it was passing traffic without a hiccup and I believe it had another 100+ days of uptime after this before I moved out. What a box. I miss that router.

Re: Unscrewed: A Story About OpenBSD

#22
post #14
post #2

Commodity hardware and general-purpose systems win every time. Honestly I wonder, who the hell buys these overpriced networking appliances? Similarly with expensive proprietary databases, or ESB systems. Are they just leftovers of a different time (note article is from 2007)? Is it a case of management buying from the salesman who bought them a nice dinner? Some place where these things generate actual value that I'm…

> Commodity hardware and general-purpose systems win every time. What's latency like on commodity hardware vs dedicated equipment? I've no experience but would expect the dedicated hardware to be better. BSD is a monolithic kernel, so you'd have the overhead of passing through the for each of the routed packets, right?

The packets never go out of the kernel, latency seems fine on my BSD router.

What word should be after "through the" in your post ?

Re: Unscrewed: A Story About OpenBSD

#23
post #14
post #2

Commodity hardware and general-purpose systems win every time. Honestly I wonder, who the hell buys these overpriced networking appliances? Similarly with expensive proprietary databases, or ESB systems. Are they just leftovers of a different time (note article is from 2007)? Is it a case of management buying from the salesman who bought them a nice dinner? Some place where these things generate actual value that I'm…

> Commodity hardware and general-purpose systems win every time. What's latency like on commodity hardware vs dedicated equipment? I've no experience but would expect the dedicated hardware to be better. BSD is a monolithic kernel, so you'd have the overhead of passing through the for each of the routed packets, right?

> through the [?] for each

BSD pf is interpreted by a kernel module in kernel mode. There is no e.g. kernel / user transition involved.

Re: Unscrewed: A Story About OpenBSD

#24
post #10
post #7

Regarding first paragraph of the story (redundant pair or routers crashing at once), it worth mentioning one possible neat trick to avoid this: When having a redundant pair of devices (routers, etc.), make sure such pair is comprised of devices made by different vendors - thinking here being that if there is a bug that triggers router crash, it's unlikely that two different vendors would be vulnerable to the same one…

No such thing as session synchronization for different vendors. You also need to configure everything for both of these vendors, which may or may not have equal feature sets.

It's hard to have independence without independence. You want separate, you're going to have to deal with separate.

Re: Unscrewed: A Story About OpenBSD

#25
post #22
post #14

Earlier quoted context omitted.

> Commodity hardware and general-purpose systems win every time. What's latency like on commodity hardware vs dedicated equipment? I've no experience but would expect the dedicated hardware to be better. BSD is a monolithic kernel, so you'd have the overhead of passing through the for each of the routed packets, right?

The packets never go out of the kernel, latency seems fine on my BSD router. What word should be after "through the" in your post ?

Sorry. "Kernel". Through the kernel. But you were there already - I hadn't twigged that there's no need to jump out of userland in the first place.

Re: Unscrewed: A Story About OpenBSD

#26
post #10
post #7

Regarding first paragraph of the story (redundant pair or routers crashing at once), it worth mentioning one possible neat trick to avoid this: When having a redundant pair of devices (routers, etc.), make sure such pair is comprised of devices made by different vendors - thinking here being that if there is a bug that triggers router crash, it's unlikely that two different vendors would be vulnerable to the same one…

No such thing as session synchronization for different vendors. You also need to configure everything for both of these vendors, which may or may not have equal feature sets.

Maybe for firewalls (synch'ing state), but for routers there's VRRP, at least.

Re: Unscrewed: A Story About OpenBSD

#27
post #7

Regarding first paragraph of the story (redundant pair or routers crashing at once), it worth mentioning one possible neat trick to avoid this: When having a redundant pair of devices (routers, etc.), make sure such pair is comprised of devices made by different vendors - thinking here being that if there is a bug that triggers router crash, it's unlikely that two different vendors would be vulnerable to the same one…

This topic is often discussed on nanog

The answers is the same as in software industry: Configuration management.

Plus it makes then harder to develop corporate cultures of deploying well known patterns that makes routers best used.

Plus it increases the surface of vulnerability for government backdoors.

In real life, there is a trend for going BSD box for cost/formation/stability/security reasons in the ISP world ... especially in non US countries that are getting kind of thinking the USA is becoming the new USSR in terms of systematic spying on every one.

These Europeans really don't get it, isn't it? They never were in the middle of cold war spying insanity where both power would puppet so called terrorists groups to make sure citizen would vote the right way under the fear.

What a bunch of stupid persons not believing that business companies are incorruptible and think they have incentive to betray them in exchange for gvt subsidies/regulations.

Re: Unscrewed: A Story About OpenBSD

#29
post #19
post #2

Commodity hardware and general-purpose systems win every time. Honestly I wonder, who the hell buys these overpriced networking appliances? Similarly with expensive proprietary databases, or ESB systems. Are they just leftovers of a different time (note article is from 2007)? Is it a case of management buying from the salesman who bought them a nice dinner? Some place where these things generate actual value that I'm…

Medium sized stable businesses. Small shops can't afford them. Large shops (Google, Facebook) hire people and use open source software.

At the moment, network switching and routing with open source products + commodity servers can't scale like Cisco and Juniper...if you want to have a fairly standard network (OSPF, BGP, etc).

Google, for example, is able to use commodity hardware because what they are routing works with a non-traditional setup...specifically a Clos network. That works for them only because their app is designed in a way that it can served up like that. It would not work for the typical kinds of traffic that you see at a normal company.

At a certain scale, barring unusual exceptions, you really do have to go with someone like Juniper/Cisco, as the ASIC based acceleration and other features aren't there in the commodity world.

There's certainly some things in progress that may change all of that, like Intel's DPDK, QuickAssist, etc.

Re: Unscrewed: A Story About OpenBSD

#30
post #5

Can that hardware keep up with crypto bitrate and firewall rules, on multiple 10gb links?

The Snabb Switch crew are starting to work on support for basic handling of packets on 100G NICs. However...

> With the new CPUs tending to increase the core count while the CPU frequency goes down, we'll get less and less "CPU cycles per packet". For example E5-2699v3 45M is 18 cores@2.3GHz. 10Gbps with 256 byte packets is achieved with 4.53Mpps, this makes ~500 clocks per packet on this CPU. Now make this 100Gbps and you get 50 clocks. You get this budget to receive, process and send the packet. 50 sounds pretty low to me.

Numbers refer to using only a single core. That particular CPU has 18 cores and 36 hardware threads.

https://groups.google.com/forum/m/#!topic/snabb-devel/pDd_uB...

Post reply on HN