Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serve2d – A protocol detecting server

github.com

Post reply on HN
Print view View original

51–60 of 77 posts

Re: Serve2d – A protocol detecting server

#51

Post by FooBarWidget »

pmelendez wrote: post #18
rakoo wrote: post #10

Earlier quoted context omitted.

The number 1 selling point of something written in Go is that it's much easier to package. The result of a compilation is a standalone binary that can be copy-pasted everywhere, as long as the architecture matches what was input at compilation time. This means: - no more having to deal with dependencies at packaging time, which makes packagers' job simpler because all they have to care is the one and only standard wa…

> no more having to deal with dependencies at runtime So, it is the same that linking the libraries statically? C and C++ has done that like since forever.

Have you ever actually tried producing a statically linked C/C++ binary? I've been programming in C/C++ for 10+ years. Static linking is a huge pain. My latest efforts have led me to create holy build boxes inside carefully controlled Docker-based environments just to be able to produce binaries that work on every Linux. With Go you can just run a single command to cross-compile binaries that work everywhere. Minimal setup required, no expert knowledge required.

Re: Serve2d – A protocol detecting server

#52

Post by marssaxman »

marcosdumay wrote: post #49
marssaxman wrote: post #38

Earlier quoted context omitted.

The reason this isn't usually done is that executable size was significant relative to storage capacity up until the early 2000s or so, and people tried to economize by deduping common parts of their executables via shared libraries / DLLs. This worked well enough to catch on, but came with an extremely high cost in added complexity, and over the years a whole layer of additional infrastructure was created in order t…

Well, you wouldn't want every executable to copy some unknown version of OpenSSL, and you'd get into all kinds of problems if you had several different versions of glibc around. But for most libraries it may really be overkill.

I see OpenSSL and libc as being effectively the "system version" as you would know it in Mac OS or Windows. It's OK to link dynamically against the operating system platform, but generally a program knows what version of the OS it is built for and expecting to be compatible with. Users know that upgrading the OS is generally worth doing but may break things.

What we lack in the unix world is a coherent division between what ought to be a very small, stable, well-understood set of fundamental system libraries suitable for dynamic linking and the vast array of utilities a developer might choose in order to get an app built without having to reinvent everything from scratch.

Upgrading libraries out from under a built, tested executable is not something we should be doing lightly, because there is no possible way to know in advance whether the apps depending on the library have succeeded in programming to the interface rather than the implementation.

Re: Serve2d – A protocol detecting server

#53

Post by ori_b »

cbhl wrote: post #23
BillinghamJ wrote: post #15

Earlier quoted context omitted.

Wouldn't a VPN or tunnelling solution be better? You pass all your traffic through a fixed port on one host, then unwrap it and use the web as though the restriction wasn't there.

Generally these well-intentioned networks block the usual VPN and tunneling solutions (think kid and a school network with a firewall that only allows FTP, HTTP, and HTTPS).

Yeah, but these well intentioned networks also often do packet inspection to block simple workarounds.

Re: Serve2d – A protocol detecting server

#54

Post by joushou »

vidarh wrote: post #44
sigjuice wrote: post #20

Earlier quoted context omitted.

VPN or tunneling solutions are unlikely to work if severe network restrictions are in place. e.g. firewalls that only allow TCP ports 80 and 443.

Nothing stops you from passing a VPN through a TCP connection on port 80 and 443. There are few protocols people haven't tunnelled IP over (DNS included...)

Actually, yes, packet inspecting firewalls do. Hence the SSH over TLS, to make things stealthy.

Re: Serve2d – A protocol detecting server

#55

Post by Retr0spectrum »

vidarh wrote: post #44
sigjuice wrote: post #20

Earlier quoted context omitted.

VPN or tunneling solutions are unlikely to work if severe network restrictions are in place. e.g. firewalls that only allow TCP ports 80 and 443.

Nothing stops you from passing a VPN through a TCP connection on port 80 and 443. There are few protocols people haven't tunnelled IP over (DNS included...)

I often have to work behind "smoothwall". AFAIK, it only allows HTTP over port 80 and HTTPS on port 443 - any other protocol on any other port gets blocked, including other protocols over port 80/443.

To bypass this, I wrote a simple ruby script to tunnel TCP connections, while adding fake HTTP headers to get through the firewall.

Re: Serve2d – A protocol detecting server

#56

Post by joushou »

ZenoArrow wrote: post #17

serve2d looks promising, can see some good uses for it. If anyone's interested, Corkscrew is an alternative solution, allows you to disguise SSH as HTTP/HTTPS traffic, useful for getting through restrictive firewalls to administer remote machines: https://github.com/elia/corkscrew

I guess the best way to find alternative solutions is to write one and have people tell you there's other like it! I genuinely didn't know there were so many doing equivalent things.

I do think serve2d is considerably more flexible than Corkscrew, though, being able to tunnel anything over TLS if needed. Or other transports, if more are added. I also let you use the same ports for other things.

Re: Serve2d – A protocol detecting server

#58

Post by joushou »

Uh, wow, the popularity just skyrocketed. It's 00:17 and I just came home from a concert, so bear with me maybe being a bit incoherent.

First of all, I'd like to thank my wife, who helped motivate me, my parents, who you know, did the thing that brought me here, my work, for paying me to browse news and occasionally code...

So, why Go? Because it was easiest that way. Try to take a look at how simple the ProxyConn thing is, which is the core of the entire thing. Go is a tool that I found suitable for the task. It also brings things like easy cross-compilation after the code has been written (Seriously, GOOS=windows go build on my mac and I get a PE32+, GOOS=darwin go build on my linux desktop and I get a Mach-O binary). You also get to interact easily with the wonderful Go infrastructure, in case you don't want to redirect to external services.

Why make something that already exists? Well, I didn't know that there were so many solutions, I just thought "Hmm, I wonder if I can make this work in a nice fashion...". That's not going to stop me from continuing development of my own, however. I have ideas I want to try, features I want to implement. It's powering everything on my own servers, and the stealthiness of SSH over TLS (NOT just SSH on port 443) have been (ab)used multiple times already.

Is my solution better than the others? Well, I of course like my own solution, but with few exceptions, I think I bring some interesting flexibility to the game. A lot fo solutions seem to be fixed for a certain purpose, such as SSH and HTTPS, usually just in their regular variants. I don't really care what you want to serve, as long as you can write down some unique bytes in the config for serve2d, or write a more complicated handler directly for serve2. Add transports if you want. Want SSH over TLS over WebSocket over IP over avian carriers? Add the transports.

I'm going to sleep now, and I hope the world haven't self-ignited before I wake up. It's really exciting and motivating to see how people react to private projects.

Re: Serve2d – A protocol detecting server

#59

Post by voltagex_ »

Retr0spectrum wrote: post #55
vidarh wrote: post #44

Earlier quoted context omitted.

Nothing stops you from passing a VPN through a TCP connection on port 80 and 443. There are few protocols people haven't tunnelled IP over (DNS included...)

I often have to work behind "smoothwall". AFAIK, it only allows HTTP over port 80 and HTTPS on port 443 - any other protocol on any other port gets blocked, including other protocols over port 80/443. To bypass this, I wrote a simple ruby script to tunnel TCP connections, while adding fake HTTP headers to get through the firewall.

Can you share that script?

Re: Serve2d – A protocol detecting server

#60

Post by pmelendez »

FooBarWidget wrote: post #51
pmelendez wrote: post #18

Earlier quoted context omitted.

> no more having to deal with dependencies at runtime So, it is the same that linking the libraries statically? C and C++ has done that like since forever.

Have you ever actually tried producing a statically linked C/C++ binary? I've been programming in C/C++ for 10+ years. Static linking is a huge pain. My latest efforts have led me to create holy build boxes inside carefully controlled Docker-based environments just to be able to produce binaries that work on every Linux. With Go you can just run a single command to cross-compile binaries that work everywhere. Minimal…

> Have you ever actually tried producing a statically linked C/C++ binary?

Many times actually, I prefer to deploy just one file whenever I can.

> just to be able to produce binaries that work on every Linux

That's a Linux design/decision thing. Linux binary compatibility is ... well ... challenging. In Windows is not hard at all (Not sure how is it in MacOS since I have worked mostly for iOS in Apple's world).

Post reply on HN