Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chisel – A fast TCP tunnel over HTTP

github.com

Post reply on HN
Print view View original

11–20 of 41 posts

Re: Chisel – A fast TCP tunnel over HTTP

#11

Post by noway421 »

Matthias247 wrote: post #4

Why TCP over websockets? You can just use the HTTP bodies as a stream in both direction. Which means the proxy just has to strip or add HTTP headers before forwarding. The overhead afterwards is 0 -> you just write to the socket.

Hmm, this choice is indeed strange, websockets still are blocked in some restrictive set ups (squid?).

But still, what is the way of doing stream in both directions? Do you mean opening multi-part form data for uploading and transfer encoding chunked for download? But that would be 2 tcp connection for 1 tcp tunnel. And I believe there's no other way to do it without the overhead of HTTP request/response headers.

Re: Chisel – A fast TCP tunnel over HTTP

#12

Post by noway421 »

badminton1 wrote: post #5

Many superlatives, which immediately raise suspicions. If you are trying to solve the problem of NAT traversal and such, I suggest you rather attempt to do this: https://en.wikipedia.org/wiki/TCP_hole_punching

this surely is great and i can't wait for a moment when someone finally comes up with a TCP Hole Punching as a Service.

vpnazure kinda does this, but has the overhead of softether vpn service on top of it... I would rather go with punching myself an ssh port.

Re: Chisel – A fast TCP tunnel over HTTP

#14

Post by noway421 »

rpcope1 wrote: post #7
badminton1 wrote: post #6

Earlier quoted context omitted.

I was asking myself the same thing. TCP implements many RFCs for congestion control, flow control, etc. Many of them might be redundant if everything is being sent over HTTP (over TCP). I would use "link conditioner" or a similar tool, simulate packet loss and see how this compares to the other software.

http://sites.inka.de/sites/bigred/devel/tcp-tcp.html Even for applications designed to tunnel traffic from the outset (OpenVPN), TCP over TCP is a mess and it's kind of a non-starter for anything that's not a toy (unless you have no other choice, like with certain mobile carriers where path MTU and CGN cause issues).

That's very interesting, thank you. I think there should be a way to control retransmission times on the client that would alleviate that problem. Probably taking out tcp connection out of kernel space and controlling the protocol in user space will allow to control the lower level tcp to work nicely with upper levels. But still, is there a possibility of a meltdown on the routers further down the line which might be controlling tcp connections? They would be routers which are not simply passing the ip packets, but working with the protocol in a more elaborate way though, either deep packet inspection or some other network mechanisms?

Re: Chisel – A fast TCP tunnel over HTTP

#15

Post by ec109685 »

buserror wrote: post #9
daurnimator wrote: post #2

Looks to be TCP over websockets; which isn't really that interesting IMO.

Yep, because many firewalls ALSO block websockets; so comparing it to crowbar for speed isn't terribly fair, as crowbar only uses POST/GETs.

They can't block websockets over https.

Re: Chisel – A fast TCP tunnel over HTTP

#16

Post by daurnimator »

ec109685 wrote: post #15
buserror wrote: post #9

Earlier quoted context omitted.

Yep, because many firewalls ALSO block websockets; so comparing it to crowbar for speed isn't terribly fair, as crowbar only uses POST/GETs.

They can't block websockets over https.

Most corp+school firewalls block https or make the user add a MITM cert (well, it's managed via active directory for company machines). Rarely do these firewalls allow websockets.

Re: Chisel – A fast TCP tunnel over HTTP

#17

Post by rusk »

noway421 wrote: post #11
Matthias247 wrote: post #4

Why TCP over websockets? You can just use the HTTP bodies as a stream in both direction. Which means the proxy just has to strip or add HTTP headers before forwarding. The overhead afterwards is 0 -> you just write to the socket.

Hmm, this choice is indeed strange, websockets still are blocked in some restrictive set ups (squid?). But still, what is the way of doing stream in both directions? Do you mean opening multi-part form data for uploading and transfer encoding chunked for download? But that would be 2 tcp connection for 1 tcp tunnel. And I believe there's no other way to do it without the overhead of HTTP request/response headers.

Not necessarily ... you would have to issue a HTTP request per uplink chunk but HTTP can use connection pooling so that does not necessarily translate to a single TCP connection [0]

Agreed it's not quite as straightforward as the parent poster suggests. I can see issues with this approach for realtime/streaming applications but for applications relying on a similar request/response flow of traffic it would do the job.

[0] https://en.wikipedia.org/wiki/HTTP_persistent_connection

Re: Chisel – A fast TCP tunnel over HTTP

#19

Post by Matthias247 »

noway421 wrote: post #11
Matthias247 wrote: post #4

Why TCP over websockets? You can just use the HTTP bodies as a stream in both direction. Which means the proxy just has to strip or add HTTP headers before forwarding. The overhead afterwards is 0 -> you just write to the socket.

Hmm, this choice is indeed strange, websockets still are blocked in some restrictive set ups (squid?). But still, what is the way of doing stream in both directions? Do you mean opening multi-part form data for uploading and transfer encoding chunked for download? But that would be 2 tcp connection for 1 tcp tunnel. And I believe there's no other way to do it without the overhead of HTTP request/response headers.

Technically HTTP represents a bidirectional stream of arbitrary data in both directions, which follows a set of headers and is optionally finished by a set of footers. This is for example quite obvious when you look at the HTTP/2 specification. There is no need for the bodies to be sent in a particular order (response body after request body) or in a particular encoding (form-data, chunked, SSE, etc.). It can be two arbitrary byte streams.

This functionality is exposed by lots of HTTP libraries, e.g. the Go, node.js or C# HTTP libraries will allow to simply read or write to a request/response body stream just like to can write to a socket. As a proxy - just copy the data from one stream to the other.

What are are probably thinking is that the browser environment does currently not allow to use HTTP for arbitrary streaming. Instead they defined some fixed use cases and encodings for these use cases. If you use SSE encoding you can't send bytes without overhead anymore and can only do streaming in one direction. But: You will get a nicer browser API for retrieving the data. Using HTTP bodies for arbitrary streaming will be allowed in future revisions of the fetch API (which e.g. give you a ReadableStream).

Other HTTP libraries also do not allow for streaming but expect the body in either direction to be a fixed length thing. This will allow to represent the body e.g. as a "string" or "byte[]" instead of a Stream from which the application has to read. Thereby the API gets simpler, but not all possible use cases are enabled.

Re: Chisel – A fast TCP tunnel over HTTP

#20

Post by dx034 »

daurnimator wrote: post #16
ec109685 wrote: post #15

Earlier quoted context omitted.

They can't block websockets over https.

Most corp+school firewalls block https or make the user add a MITM cert (well, it's managed via active directory for company machines). Rarely do these firewalls allow websockets.

I don't think that any company can still block https, there are not many pages left you could use.

Installing a MITM is probably more common. Although I don't understand the benefit of then blocking ws, if you can read the traffic anyway?

Post reply on HN