Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choosing Go to implement the new GOV.UK router

gdstechnology.blog.gov.uk

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Choosing Go to implement the new GOV.UK router

#31

Post by intranation »

buro9 wrote: post #21
justincormack wrote: post #2

As they are Using Nginx already interested to know if they ruled out openresty or did not consider it.

I'm a big fan of Go and we use it internally, but I too wonder why nginx + Lua wouldn't be the chosen path. If it's good enough for Cloudflare it probably is good enough here. Besides, Varnish also would have excelled (yes I read the first post in the series) if they had just stored the configuration in a saner format and then output the required VCL when the config changed and reloaded on the fly. Both of the techno…

Openresty and Lua are great, but Go is a more generally applicable language for building web services. So when taken in that context Go makes more sense if we look at it as a learning exercise as well as a new part of the stack.

Re: Choosing Go to implement the new GOV.UK router

#32

Post by oblio »

floops wrote: post #18

A lot of the comments here seem to be missing something quite important: Scala and Go are not similar languages. Scala is a large and complex language, encompassing multiple programming idioms. Teams programming Scala (as with C++ and many other "large" languages) often have difficulties nailing down what portion of the language they are going to use. "Java-without-semicolons" all the way through to idiomatic, functi…

"Big is beautiful" versus "small is beautiful". Some people like Ruby, others like Python.

Hell, some people like Perl! :p

Re: Choosing Go to implement the new GOV.UK router

#33

Post by JazCE »

intranation wrote: post #31
buro9 wrote: post #21

Earlier quoted context omitted.

I'm a big fan of Go and we use it internally, but I too wonder why nginx + Lua wouldn't be the chosen path. If it's good enough for Cloudflare it probably is good enough here. Besides, Varnish also would have excelled (yes I read the first post in the series) if they had just stored the configuration in a saner format and then output the required VCL when the config changed and reloaded on the fly. Both of the techno…

Openresty and Lua are great, but Go is a more generally applicable language for building web services. So when taken in that context Go makes more sense if we look at it as a learning exercise as well as a new part of the stack.

Scala wasn't a learning exercise then?

Re: Choosing Go to implement the new GOV.UK router

#34

Post by orf »

csmuk wrote: post #20
intranation wrote: post #15

Earlier quoted context omitted.

We did have some configuration - it was stored in Varnish and Nginx. The time cost of shipping new redirects when content is moved or new routing configuration for new tools coming online is non-negligible, particularly as we're in the middle of moving 300+ government agencies onto the GOV.UK platform. Also having all those redirects, friendly URLs, and application mappings in static configuration makes self-service…

They're incredibly easy to build. I notice you use puppet. That's enough tooling to push configuration out and restart services for nginx or apache+mod_proxy. You can break configuration out into separate files per agency if you need to. Varnish is different as that requires restarting. We do the same with commercial kit (Riverbed) with over 140 HTTP application endpoints and that is higher friction than the equivale…

and you are not a government agency delivering digital services to 60+ million people

Re: Choosing Go to implement the new GOV.UK router

#35

Post by txutxu »

Something is wrong with this statement:

    "Scala is great for performance, but quite bad at resource usage"
Maybe it's directly related to the next statement:

    "No-one in the core GOV.UK team had a deep knowledge of Scala, and particularly how the old router worked"
But the first statement, literally, does not make sense on its own.

Re: Choosing Go to implement the new GOV.UK router

#36

Post by girvo »

thu wrote: post #7

That seems weird to me: * No one having a deep knowledge of Scala is mentioned as a problem, but they will learn Go. * They rule out a language because of its syntax.

I like Erlang, but in shops I've worked with that have your "average" dev in it, it's syntax and semantics would be something they struggle with (if it ain't PHP, they'll struggle with it...). Unfortunately I don't blame them. That said, I think if they truly needed to build a somewhat stateful concurrent router, Erlang is a better choice (solely because it's battle tested in those highly concurrent workloads). Picking tech for big things like this when you have to balance a teams knowledge with the right tool is a hard problem in my experience.

Re: Choosing Go to implement the new GOV.UK router

#37

Post by walshemj »

csmuk wrote: post #14
daliusd wrote: post #13

Earlier quoted context omitted.

It does not look that bad actually. Your list lacks Oracle, Microsoft technologies to really complain about inefficient usage of public money. I would love if my country used open source more.

Agree entirely there! However it doesn't excuse people working on a web site funded by the taxpayer using it as a technology playground.

you'd rather £900-£1000 /per diem consultants from Captita where doing a worse job?

Re: Choosing Go to implement the new GOV.UK router

#38

Post by girvo »

JazCE wrote: post #28
floops wrote: post #18

A lot of the comments here seem to be missing something quite important: Scala and Go are not similar languages. Scala is a large and complex language, encompassing multiple programming idioms. Teams programming Scala (as with C++ and many other "large" languages) often have difficulties nailing down what portion of the language they are going to use. "Java-without-semicolons" all the way through to idiomatic, functi…

You're right, but i think the point you're missing (and i don't mean this in a rude way) is that they had a system built in scala, somehow, but now no one on the team knows scala. They lack an architectural oversight, that they should really be using one(ish) language rather than a big mix of languages so that this doesn't occur again when no one on the team knows GO because the project leads taken a new job.

Retention. Also, I'd suggest that your average dev will learn Go quick enough to work on this router quicker than they'll learn Scala; I prefer Scala myself, though! It's a balance between proper tech choice, correct management choices moving forward, and dealing with management mistakes (like letting someone build the router in Scala when the rest of the team don't know it) from the past.

A crappy PHP dev will learn Go a hell of a lot quicker than Scala, IMO. A safer choice, for their team and position, IMO.

Re: Choosing Go to implement the new GOV.UK router

#39

Post by dylz »

floops wrote: post #30
csmuk wrote: post #25

Earlier quoted context omitted.

No but updating static routes via puppet/ansible and reloading is a piece of piss. If your site is volatile enough that the routing needs to change that often then there is something wrong either with your information architecture or your development process.

Or maybe you're trying to deal with a huge legacy content migration of several hundred different government agencies and want to deploy several times a day without waiting for 10 minutes for nginx to load 1MB of config each time?

Or the fact that having multi-megabyte configuration files in the first place is in itself somewhat of a horror...

Re: Choosing Go to implement the new GOV.UK router

#40

Post by csmuk »

floops wrote: post #30
csmuk wrote: post #25

Earlier quoted context omitted.

No but updating static routes via puppet/ansible and reloading is a piece of piss. If your site is volatile enough that the routing needs to change that often then there is something wrong either with your information architecture or your development process.

Or maybe you're trying to deal with a huge legacy content migration of several hundred different government agencies and want to deploy several times a day without waiting for 10 minutes for nginx to load 1MB of config each time?

2 front end nginx boxes s01-s02:

1. take s01 out of pool.

2. migrate config on s01 to new config.

3. put s01 back in pool.

4. take s02 out of pool

5. migrate config on s02 to new config.

6. put s02 back in pool.

Ansible can handle this quite happily. Scales up to any number of boxes. For an (n-1)/n capacity reduction during deployment.

I've got a 450k apache config somewhere that takes Also if you have THAT much config, something is wrong with your information architecture (see my other points).

Post reply on HN