Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FastCGI: 30 years old and still the better protocol for reverse proxies

agwa.name

Post reply on HN
Print view View original

81–90 of 107 posts

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#81

Post by Oxodao »

yesbabyyes wrote: post #78
apitman wrote: post #35

> That said, using a vintage technology has some downsides. It was never updated to support WebSockets With widespread browser support for WHATWG streams, it's pretty easy to implement your own WebSockets over long-lived HTTP requests. Basically you just send a byte stream and prepend each message with a header, which can just be a size in many cases. Advantages over WebSockets: * No special path in your server layer…

Please be aware that there is a web standard for this since quite some time. See server-sent events and the EventSource interface: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource

That can be used with https://mercure.rocks :)

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#82

Post by xorcist »

nostrademons wrote: post #3

This is quite an interesting article for its omissions. I remember the great FastCGI vs. SCGI vs. HTTP wars: I was founding a Web2.0 startup right at the time these technologies were gaining adoption, and so was responsible for setting up the frontend stack. HTTP won because of simplicity: instead of needing to introduce another protocol into your stack, you can just use HTTP, which you already needed to handle at th…

I don't think that is how it happened. Yes, there was a SCGI/FastCGI chism, but it was mostly the Python ecosystem that used SCGI and the rest of the world was on FastCGI. Unless you were PHP, in which case you were on mod_php because it was an unmovable juggernaut.

Apache had a FastCGI module early on, but it received little love and was not that widely used. For many people, FastCGI was synonymous with nginx and lighttpd because these webservers came with support out of the box (nginx later got modules just as Apache).

When PHP finally got PHP-FPM, that gigantic ecosystem slowly started moving, sometime in the late 00s, and then FastCGI really took over. Almost. Because at the same time, the cloud era started and brought the "just use HTTP bro" mindset. Amazon has always used HTTP internally since the 90s and I would guess that probably carried over to AWS?

So nowadays, PHP still being a silent juggernaut, is now mostly on FastCGI, while most other have moved to cloud era standards and use HTTP. Go, for example, matured at this time and all the tutorials use straight HTTP proxying.

Yes, FastCGI is the much more robust standard, but you will encounter friction if you use it on your cloud native application. For regular servers and VMs it is still common.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#83

Post by xorcist »

The mystery is why uWSGI isn't more widely used. Perhaps the name does not help. It has little to do with WSGI just as FastCGI is unrelated to CGI.

It is a tiny binary protocol, with frames just as FastCGI. The reference server works with several languages, I've used it over the years mostly with Python but also Ruby and Perl. It is a small C executable with all the practical features one need for web hosting: Draining backends, autoscaling, logging, chrooted backends, everything.

Very few FastCGI servers are this mature. Unlike FastCGI, it has been extended to support websockets and async.

I have used it in production at several places for many years and have nothing but praise for it. It feels like this weird unknown secret for web operations. Unfortunately, it sees lesser use now in the cloud era, and development seems to have all but stopped. It still works and is still reliable but the writing is probably on the wall. However nothing comes close in terms of speed, simplicity, and features.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#85

Post by zokier »

Does anyone remember mongrel2? A neat web server that used zeromq for backend communication (based on scgi). I always felt it had huge potential. It was also kinda famous in the early days of HN, back in the days when RoR was in vogue etc.

https://hn.algolia.com/?q=mongrel2

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#86

Post by alfiedotwtf »

nostrademons wrote: post #3

This is quite an interesting article for its omissions. I remember the great FastCGI vs. SCGI vs. HTTP wars: I was founding a Web2.0 startup right at the time these technologies were gaining adoption, and so was responsible for setting up the frontend stack. HTTP won because of simplicity: instead of needing to introduce another protocol into your stack, you can just use HTTP, which you already needed to handle at th…

To summarise, nginx!

There was a small window where everyone was trying to move off Apache/mod_perl etc, coming up with all sorts of ways to talk to the backend faster… but then nginx walked into the chat and killed the 10k problem along with having its easy but fancy upstreaming, and that was that.

Nginx made horizontal scaling a winch, and because of that rewriting your backend to handle HTTP and FastCGI etc was more effort than it was worth.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#87

Post by assimpleaspossi »

opello wrote: post #70
assimpleaspossi wrote: post #67

Earlier quoted context omitted.

Your last sentence is exactly what I said.

You didn't though. You may have intended to? The article points out that HTTP and FastCGI are both options for reverse proxies to communicate to the downstream server. I didn't find a reference to them being interchangeable outside of that context. If there is or was one please quote it.

I was responding to the comment, not the article.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#88

Post by assimpleaspossi »

anon7000 wrote: post #69
assimpleaspossi wrote: post #67

Earlier quoted context omitted.

Your last sentence is exactly what I said.

The article is really exclusively about the reverse proxy server to server use case, not client to server. The title even says it.

I responded to the comment, not the article.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#89

Post by assimpleaspossi »

scottlamb wrote: post #65
assimpleaspossi wrote: post #56

Earlier quoted context omitted.

>>FastCGI is better than HTTP for these things. FastCGI and HTTP are at two different levels. HTTP is for data transfer from, say, a browser and a server. FastCGI is for handling that data between the server and an application. Just now I glanced at the article and it seems the author writes in a confusing way to imply that HTTP and FastCGI are interchangeable and they are not. fwiw, I used fcgi for a decade for all…

>> FastCGI is better than HTTP for these things. > FastCGI and HTTP are at two different levels. HTTP is for data transfer from, say, a browser and a server. FastCGI is for handling that data between the server and an application. Just now I glanced at the article and it seems the author writes in a confusing way to imply that HTTP and FastCGI are interchangeable and they are not. That might be just you. The article…

I didn't quote anything from the article. I was responding to the comment, not the article.

Re: FastCGI: 30 years old and still the better protocol for reverse proxies

#90

Post by xorcist »

somat wrote: post #30
shevy-java wrote: post #14

I'd love for CGI to be updated, kind of merging what works and not really caring about what does not work. Getting a .cgi file to work on Linux is really easy. Naturally you get more leverage with e. g. rails, but there is also a lot more complexity and I really hate intrinsic complexity.

CGI and FastCGI are two different things in two different domains. Well the domains are not that different but enough that CGI solves a real problem and makes sense and FastCGI does not. CGI is the interface between a HTTP transaction and a process. It answers the question "How do we turn a HTTP request into executing a process?". FastCGI answers the question of "How do we turn a HTTP request into a FastCGI request".…

Your question is why we have http servers and http middleware at all. Running an application directly on an endpoint makes sense for embedded application, but for everything else you will end up with a separate http terminating layer for a number of reasons.

It makes maintenance so much easier, when you can scale request backends independently of request frontends. It makes sense that the application doesn't have access to TLS keys and can't bind end user facing ports directly. The day you want a separate access log from the application log, and separate instrumentation of the frontends because of some hard to track down bug you will be thankful it is there.

So, you want to deploy some sort of proxy. Will you run FastCGI, SGI, WSGI, uWSGI, plain http, or a number of proprietary load balancer protocols? If you choose http you have to take care to wash your headers and be very careful with pipelining so front and back end expect the same behaviour (which is true for several of the proprietary protocols too). Otherwise there can be catastrophic security implications.

It is probably true that FastCGI was designed as a persistent CGI. But as soon as you have persistence, the http server needs to forward transactions to the persistent backend, and what you have built is a proxy. There is no way around it.

Post reply on HN