Live data from Hacker News

An ex-Googler’s guide to dev tools

about.sourcegraph.com

181–190 of 219 posts

Re: An ex-Googler’s guide to dev tools

#181
very interesting article. how do people stay updated with latest tools outside of google's ecosystem and do they feel like they are missing out.

lastly I am just curious to know why don't companies adopt opensource tools that are being used everywhere so skills can be transferable easily

Re: An ex-Googler’s guide to dev tools

#182

For me, logs processing system and Dremel were probably the most marvelous tools that I've ever seen in my SWE career. Millions of machines are producing trillions of logs records everyday. Then Dremel runs some arbitrary query over those records in an order of minutes (seconds for simple queries), without indexing.

There are a few oss dremel alternatives with Clickhouse being my favorite

To me their absolute best technology edge is still their storage system (colossus)

Re: An ex-Googler’s guide to dev tools

#183

Honest opinion on FAANG tools is that the big ones are now antiquated and slowing these big tech companies down. Eventually they will need to rip out all these tools that once gave them an edge and replace them, but they cant bc of how much depends on them, and whala, you have a large failing slow company

Right if only they could get on such cutting edge tech as npm, cmake or just gnu make with a pile of bash like most oss projects out there

Re: An ex-Googler’s guide to dev tools

#185

Earlier quoted context omitted.

Yes please, Github PRs are much, much better than Gerrit or Ritveld (urgh), and the +1/+2 quirks of it No, I'll take Github PRs any day. They could be better, of course.

Stacked reviews are the crucial thing missing from GitHub pull requests. Truly, once you get used to working wit hthose, going back is intolerable, to the point where I now have a hacked together workflow where I live in git rebase -i and have it automatically push specially named branches for each commit in a stack.

At my place of work, we've sort of implemented that feature by using fixup or squash commits. That is, if a comment is made on a diff line in a PR, use git blame to determine which commit was responsible for that line of code, update the line and commit it with a commit title like: fixup! . If we want to update the commit message, we use "squash! " and put the updated commit message in the commit message body (using the --allow-empty switch when committing so that we don't have to change any code to make a commit).

Then, we can run git rebase -i --autosquash --keep-empty to handle applying the changes requested to the correct commits at the end of the review process.

Re: An ex-Googler’s guide to dev tools

#186
post #175
post #11

> Many years ago, I did a brief stint at Google. A lot has changed since then, but even that brief exposure to Google's internal developer tools left a lasting impression on me. Perhaps the operative phrase being "Many years ago". I currently work at Google, previously I worked at Square. Of the two, I generally prefer the OSS and off-the-shelf tooling at Square. Some things really are better at Google (code search a…

> I currently work at Google, previously I worked at Square. Of the two, I generally prefer the OSS and off-the-shelf tooling at Square. I wrapped a 2-year stint at Facebook recently and my experience was similar to yours. Some tools were amazing and cool and I really appreciated how they evolved over time to manage huge amounts of resources. But most were subpar when compared to OSS tooling - outdated, deprecated fu…

You have to make some allowances for first movers. The designs of new tools are often ill conceived because they have no templates to steal ideas from. But the emotional investment often keeps them from switching to better tools because We’ve Always Done Things This Way.

The thing that bugs me most about my company’s tools are the ones that are enough younger than OSS solutions that someone either didn’t look very hard, or didn’t want to find anything (so they could write their own). Other priorities come along and those tools eventually can’t keep up with OSS alternatives, but the apologists take over.

You will not continue to get accolades for tools you wrote three years ago. The only “value” you derive is the time and effort it saves, offset my the effort expended. The time and effort expense of external tools are often lower. And, when the tools are annoying, you can commiserate with your coworkers instead of being the target of their criticisms. Which is often under-appreciated.

Re: An ex-Googler’s guide to dev tools

#187

Earlier quoted context omitted.

Hasn't pcon been deprecated for a year and a half now? :P More pixels != to more information. The visuals and varieties in ways to present data in grafana always seemed more intuitive for me, being able to lay stuff out in different ways was great. I worked on a G-on-G project at Google and used both grafana and viceroy. I never had an issue with grafana on mobile, but I was never was able to see viceroy on a mobile…

On viceroy if you unintentionally put 2000 lines on a graph, you get a visual mess. If you do that on grafana your tab will hang for ten minutes. Big difference.

True, but that only happens when setting up graphs. Graphs are not changed as often as they are viewed.

I'm sure viceroy is more performant, and I'm not denying that, I may also be scarred a bit because I found writing mash queries an arcane nightmare, and those are related to viceroy.

The default (rpc latencies and etc) views and metrics you get are amazing however.

Re: An ex-Googler’s guide to dev tools

#188
post #156

Earlier quoted context omitted.

Google’s internal code search is far ahead of GitHub, Gitlab, and even sourcegraph. But you can use it, because they open-sourced it. I don’t know why more people don’t use it. https://kythe.io/

Author of the post here. I would say that technically Kythe is the open-source version of part of Google's internal code search (specifically, the component that provides precise code navigation), but it doesn't include the search index or the UI. So it's not on its own an end-user product. That having been said, I love Kythe, and we've actually considered using it as a semantic backend for Sourcegraph (and still mig…

(kythe googler here)

Like beliu said, the Kythe schema is far richer; it has fully abstract semantic layer in the graph, and is a superset of what can be represented with LSIF. It's not tied to specified text regions -- there are representations of symbols/functions/classes/variables/types that do have pointers to/from text regions.

Note that because of the richness and abstractness, it's theoretically feasible to drive much more than code navigation from the Kythe graph.

And yes, the open source is just part. The large scale pieces are basically (1) do instrumented build (2) run through Kythe indexers (3) post-process output for serving.

The Kythe OSS project offers solutions for (2) for C++/Java/Go/Typescript/protobuf (and early Rust support). We do have plans to open source support for at least some other languages at some point in the future. (Hedging as best I can here.) Note that the best candidates for Kythe indexing are those languages that admit solid static analysis.

(1) is inextricably tied to the build system. Bazel support should be nearly turnkey; other systems require more (maybe significantly more) work.

There's not-full-scale support for (3) available. (Clearly we use something far more sophisticated internally.) While we'd like to see this fleshed, expansion of that will depend on non-trivial community contributions.

Re: An ex-Googler’s guide to dev tools

#189
post #61
post #8

Isn't code search more table stakes nowadays? Both github and gitlab have it. There isn't anything said about end to end, integration or functional testing here. I'm in a world where everyone hacks their own system together onto the same runtime, leading to some wonky outcomes and lots of operational support. Would be interesting if there was a 'google' way to do it.

Google used to offer code searching for public repos but then, expectably, they abandoned it. Now it's only used for google-owned/run projects. But when it was around it was really amazing. Github's code search is quite lame in comparison. I often just run ripgrep locally instead.

regarding google-associated projects: that's true enough, but note that you can e.g. navigate a version of LLVM at

https://cs.android.com/android/platform/superproject/+/maste...

Re: An ex-Googler’s guide to dev tools

#190

Earlier quoted context omitted.

On viceroy if you unintentionally put 2000 lines on a graph, you get a visual mess. If you do that on grafana your tab will hang for ten minutes. Big difference.

True, but that only happens when setting up graphs. Graphs are not changed as often as they are viewed. I'm sure viceroy is more performant, and I'm not denying that, I may also be scarred a bit because I found writing mash queries an arcane nightmare, and those are related to viceroy. The default (rpc latencies and etc) views and metrics you get are amazing however.

You can also get tons of lines when modifying group by expressions, which you can do ineffectively.

Mash is indeed arcane, although gmon (viceroy) makes it worse than usual.

Post reply on HN