Live data from Hacker News

Google's ‘Gopher Team’

wired.com

11–20 of 40 posts

Re: Google's ‘Gopher Team’

#11
post #5

If code doesn’t receive constant love it turns to shit, I would prefer "If bad code doesn't receive constant love it turns to shit" I am aware of thousands of examples of heavily used commercial software that hasn't been touched in 5, 10, 15, even 20 years because it just works and always has. Properly designed and written scalable software can last indefinitely with little or no modification, even through geometric…

Properly designed and written scalable software can last indefinitely with little or no modification

The problem is that what may be "proper" today (thinking, say, 5-10 years out) may fail the test in 10-15 years. One has to design software under some constraints, usually defined by the problem boundary and thus it by definition cannot be infinitely scalable and flexible.

Also, the problem is the code ends up being maintained by programmers of varying levels of competency. Even if competency is not a problem, the varying styles inadvertently lead to spaghetti and inconsistent code.

Re: Google's ‘Gopher Team’

#13
post #12

I really don't understand why dl.google.com isn't just running Nginx or some web server. It's just serving files. Why does it need software written in house?

Google is the only company on the planet with anything approaching its scale requirements. It's entirely possible that even nginx breaks down at their level of speed/concurrency/distribution.

Re: Google's ‘Gopher Team’

#14
post #10

No they don't. One guy started a major rewrite of a broken piece of infrastructure, and "the Go team [which he is part of] now regularly volunteers to help other teams with small projects", so as to learn more about how Go can replace and augment existing systems. This headline is so far beyond journalism as to be a joke. It's just making shit up for the sake of having a "story". I can imagine it now. They heard "gop…

Yep. I wouldn't surprised to see this article come from some other outlet; I did not think it's already time to add Wired to the "yellow media" bucket (though last week's article about LLVM being the last thread holding Google and Apple together just about did it)

Wired really depends on the author, IMO. I still enjoy Threat Level

Re: Google's ‘Gopher Team’

#15
post #12

I really don't understand why dl.google.com isn't just running Nginx or some web server. It's just serving files. Why does it need software written in house?

Nginx was built with 'single server' architecture in mind while Google generally operates on a 'distributed everything' architecture. This involves stuff like distributed file systems/data stores. My guess is it's not compatible with the single file system interfaces of Nginx. Also note that dl.google.com serves stuff on a massively bigger scale than the majority of the rest of the internet. On that scale I'd guess they face some rather uncommon challenges and bottlenecks that can only be addressed by custom software.

Re: Google's ‘Gopher Team’

#16
post #8

Have any heavy C++ devs switched to go? What's the major advantage/compelling reason to switch? It seems to me that Go is more comparable to Python/Ruby and Java on the Web, but maybe I'm wrong.

I'm not a heavy C++ developer and only use it to augment other languages when performance demands it, but not having to deal with memory management and the problems it can create when binding to another language is a nice benefit. The only drawback with Go though for my usage is when using on Android, since it has to be compiled statically when used with the NDK.

Re: Google's ‘Gopher Team’

#17
post #5

If code doesn’t receive constant love it turns to shit, I would prefer "If bad code doesn't receive constant love it turns to shit" I am aware of thousands of examples of heavily used commercial software that hasn't been touched in 5, 10, 15, even 20 years because it just works and always has. Properly designed and written scalable software can last indefinitely with little or no modification, even through geometric…

It doesn't work that way. The underlying assumptions change. The program can be perfect but still not matching what becomes necessary in a few years. The example they write about is exactly such a kind. Sometime in 2007 it was apparently slow to access files from the distributed storage on Google. Therefore, a Python script was written to fetch these files to some local disks. Python was obviously enough since that process was slow. Then, another program, written in C++ was to deliver the files from the local disks to the clients. The C++ program was then "sub-optimally" maintained, simply because it wasn't on the critical path for the company. There weren't even the good test cases to verify correctness of the changes.

Still, until 2013 a lot of assumptions based on which the program was written changed completely: Accessing files from the distributed storage became faster. The typical computer which is to deliver the data seems to have much more free RAM, allowing using RAM for caching purposes instead of the local hard disk. The nature of the load changed so much that accessing the local hard disk became the bottleneck anyway. Which is not so surprising: one disk seek is still around 10 ms, which means as soon as you can't deliver something from RAM you can deliver only up to 100 items from the disk per second, when they are not in the same block of data.

Don't tell me you have to write the program for "all possible assumptions." Unless you have unlimited time, you write it to solve the problem under existing assumptions. Once they change, you change the implementation.

Moreover, to achieve the changes related to the underlying assumptions, Go wasn't needed in the current example, but it was the most convenient approach for Brad, who is by the way obviously an extraordinary programmer.

What I agree with you is that if the program is bad even under the initial assumptions, it's certainly something that should be observed and tracked by the good manager: he must know that he can't rely on such program and that whenever the program gets any heavier load, the problems will happen.

Re: Google's ‘Gopher Team’

#18
post #5

If code doesn’t receive constant love it turns to shit, I would prefer "If bad code doesn't receive constant love it turns to shit" I am aware of thousands of examples of heavily used commercial software that hasn't been touched in 5, 10, 15, even 20 years because it just works and always has. Properly designed and written scalable software can last indefinitely with little or no modification, even through geometric…

Or just deploy the half-assed solution, which will get the job done and allow your limited engineering staff to focus on higher priorities. Then, a decade later, when the company is worth hundreds of billions, you can rewrite it from scratch the proper way.

See also: A Tale of Two Bridges http://hintjens.com/blog:16

Re: Google's ‘Gopher Team’

#19
post #8

Have any heavy C++ devs switched to go? What's the major advantage/compelling reason to switch? It seems to me that Go is more comparable to Python/Ruby and Java on the Web, but maybe I'm wrong.

* No circular dependencies (big win here)

* Package importing is easy and a first-class language structure

* Fast compiles (related to no circular dependencies)

One of the more controversial features is no inheritance -- only composition. It's an interesting break from vertical/horizontal inheritance schemes.

Re: Google's ‘Gopher Team’

#20
post #12

I really don't understand why dl.google.com isn't just running Nginx or some web server. It's just serving files. Why does it need software written in house?

dl.google.com isn't just a place to dump files to the public web, it's also got complicated ACL schemes that define restrictions on data access. dl.google.com is used internally.
Post reply on HN