Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everybody gets WebSockets

blog.cloudflare.com

Post reply on HN
Print view View original

41–50 of 78 posts

Re: Everybody gets WebSockets

#41

Post by williamstein »

nulltype wrote: post #36

I'm a little confused by the pricing page: https://support.cloudflare.com/hc/en-us/articles/200169466-C... It's not clear to me what a "low" or "high" volume of connections is, and if that is number of connections or throughput.

The rest of that page suggests they haven't decided yet. They say "Barring abuse or attack, we will not impose limits errors for any application without contacting the customer. Customers whose usage claims a disproportionate percentage of resources for their current plan level may be asked to upgrade to the plan level that matches their needs." I currently have a $200/month business subscription (for https://cloud.sagemath.com), which was required for websockets before today, and will very likely switch to a $20/month pro subscription. We typically have up to about 500 concurrent connections on our site. Incidentally, we switched to using CloudFlare about a month ago after being hit by a DDOS attack (400MB/s incoming traffic).

[Edit: actually, we're not switching since only the business plan provides "Advanced DDoS protection - layer 3 and 4", which was why we are using CloudFlare in the first place.]

Re: Everybody gets WebSockets

#42

Post by jhgg »

Neat. We've been using CF websockets for some time now on the Enterprise plan. If you're intending to use CF websockets, be prepared for random (and potentially massive) connection drops, and be sure you're architected to handle these disconnects gracefully. Cloudflare rolling restarts have caused hundreds of thousands of websocket connections to be disconnected in a matter of minutes for us. If you plan to operate at that scale, make sure you're able to tolerate the thundering herd of reconnecting clients as cloudflare disconnects them.

Re: Everybody gets WebSockets

#43

Post by lostcolony »

mark242 wrote: post #38
markbnj wrote: post #29

I get that the overhead of an individual message over an individual connection is lower, but for that efficiency you give up layer 7 routing capability, make load balancing difficult, have more long-lived connections to your servers. Does HN generally feel like these are worthwhile tradeoffs?

Absolutely yes. Connections are very very cheap to maintain, and very very expensive to build up and break down. Even in environments with load balancers, stateful firewalls, etc., it is much easier to keep a socket open than it is to create a new one. If you're doing any kind of realtime data, websockets are the way to go. It is way, way easier to do stream processing and deal with backpressure than it is to have a…

ELBs support websockets fine with TCP/Secure TCP listeners.

What they -don't- support at that layer is sticky sessions, ip hashing, or similar; when a connection drops, you could reconnect to -any- instance behind the ELB. You have to engineer for that.

They also don't support Socket.IO if you have more than one instance behind the ELB (as, presumably, you do). This is because Socket.IO doesn't just open a websocket connection, and on failure fall back to something else; instead, it makes a REST call to the backend to see what connections it supports, then makes the websocket call. Both of these calls have to hit the same Socket.IO instance, because they're tied together with a session identifier. If they hit separate ones, then Socket.IO rejects the websocket, because that instance has never seen that session identifier before. But this is particular to Socket.IO's implementation, and not websockets themselves.

Re: Everybody gets WebSockets

#44

Post by seabee »

chinathrow wrote: post #34
izak30 wrote: post #30

Earlier quoted context omitted.

Pretty trivial to get a new IP address these days.

IP enumaration is also pretty trivial. I guess that what attackers might be using to "unhide" domains. Scan the net and request the required domain name?

That won't work if the origin firewall allows only CloudFlare IP ranges.

Re: Everybody gets WebSockets

#45

Post by justifier »

kefka wrote: post #28
nilved wrote: post #24

Not Tor users

Precisely. Because of CloudFlare's position regarding Tor users, along with the erroneous idea that an identity is an IP address, these changes do nothing for the Tor user and developer community at large. Tor is used for more than just routing around censorship. I use it to create a seamless network of all my computers all via hidden services. So every machine has a "hidden service domain name" of [hash].onion . Kno…

> Knowing all the hashes of my machines means I can then use all my machines as a computing cloud.

went looking through your github and blog for some code andor a writeup

any plans to post? god of magic :p

Re: Everybody gets WebSockets

#46

Post by kefka »

justifier wrote: post #45
kefka wrote: post #28

Earlier quoted context omitted.

Precisely. Because of CloudFlare's position regarding Tor users, along with the erroneous idea that an identity is an IP address, these changes do nothing for the Tor user and developer community at large. Tor is used for more than just routing around censorship. I use it to create a seamless network of all my computers all via hidden services. So every machine has a "hidden service domain name" of [hash].onion . Kno…

> Knowing all the hashes of my machines means I can then use all my machines as a computing cloud. went looking through your github and blog for some code andor a writeup any plans to post? god of magic :p

Yeah, I was making some big changes on my blog, and things kind of exploded in my face. My fault, it wasn't terribly critical so I nuked and reinstalled.

Just install Tor as you normally would, and turn on Hidden Services from port 22 to port 22 (for SSH). Keep track of the generated Tor onion hostname when you restart with Hidden Services enabled.

______________________________________

But here's the 'magic' part how to get .onion resolution across a Linux system:

get the following packages (Ubuntu, Debian)

    sudo apt-get install tor iptables dnsmasq dnsutils
Add the following to the /etc/tor/torrc file

    VirtualAddrNetworkIPv4 10.192.0.0/10
    AutomapHostsOnResolve 1
    TransPort 9040
    DNSPort 53
    DNSListenAddress 127.0.0.2
Restart TOR

    sudo service tor restart
Edit /etc/dnsmasq.conf and add the following:

    listen-address=127.0.0.1
    resolv-file=/etc/realresolv.conf
    server=/onion/127.0.0.2
Make a new file, called /etc/realresolv.conf . Add this in the file:

    nameserver 107.170.95.180 (or whatever nameserver you choose)
    nameserver 8.8.8.8
Restart DNSmasq:

    sudo service dnsmasq restart
Run the IPtables firewall update for redirection

    sudo iptables -t nat -A OUTPUT -p tcp -d 10.192.0.0/10 -j REDIRECT --to-ports 9040
Also, this script must be run at every boot, so add this in /etc/rc.local, ABOVE the "exit 0"

    /sbin/iptables -t nat -A OUTPUT -p tcp -d 10.192.0.0/10 -j REDIRECT --to-ports 9040
________________________________________________

Re: Everybody gets WebSockets

#47

Post by CiPHPerCoder »

mmaunder wrote: post #26
billmalarky wrote: post #23

Earlier quoted context omitted.

The benefit is this will allow people who previously had to reveal their origin IP (that is non business class users) in order to support websockets will now be able to hide origin behind cloudflare. Cloudflare can't protect you from DOS attacks if the attacker knows the IP address of your origin.

http://viewdns.info/iphistory/

http://viewdns.info/iphistory/?domain=paragonie.com

Nope.

Re: Everybody gets WebSockets

#48

Post by matt4077 »

slrz wrote: post #35
jephir wrote: post #27

Earlier quoted context omitted.

The biggest thing for me is that you can use secure WebSockets (wss://) without having to setup TLS on your origin server. This greatly improves the ability to establish WebSocket connections across proxies.

So what you want to do is fooling clients into believing there is transport encryption while actually there is none and the communication with the origin server happens in the clear?

From what I remember, CF actually requires an ssl certificate, but they'll accept a self-signed one (because they already validate ownership themselves.

Re: Everybody gets WebSockets

#49

Post by [deleted] »

kefka wrote: post #28
nilved wrote: post #24

Not Tor users

Precisely. Because of CloudFlare's position regarding Tor users, along with the erroneous idea that an identity is an IP address, these changes do nothing for the Tor user and developer community at large. Tor is used for more than just routing around censorship. I use it to create a seamless network of all my computers all via hidden services. So every machine has a "hidden service domain name" of [hash].onion . Kno…

[deleted]

Re: Everybody gets WebSockets

#50

Post by CyberDildonics »

markbnj wrote: post #29

I get that the overhead of an individual message over an individual connection is lower, but for that efficiency you give up layer 7 routing capability, make load balancing difficult, have more long-lived connections to your servers. Does HN generally feel like these are worthwhile tradeoffs?

Maybe you should ask the people using your web site if they care.
Post reply on HN