Live data from Hacker News

GitHub Insights

github.com

41–50 of 58 posts

Re: GitHub Insights

#41
CEO of Code Climate here. We have a product Velocity (https://codeclimate.com/) which offers what we call Engineering Intelligence. There's some great discussion about the value and appropriate use of data in software engineering, so I thought I'd chime in.

What we've seen is that engineers inherently want to be productive, and are happiest when they can work friction-free. Unfortunately, it can be quite difficult to get visibility into roadblocks that slow down developers (e.g. overly nitpicky code review, late changing product requirements, slow/flaky CI), especially for managers who are one or two levels removed from programming. These are situations where data-backed insights can be helpful for diagnosis.

After diagnosing issues, with data or simply qualitative insights from a retrospective or 1:1, we also see teams sometimes struggle to set goals and achieve desired improvements. A common theme is the recurring retrospective item that people agree is important but doesn't seem to be resolved. When it comes to implementing improvements, data can be useful to make objectives concrete and make progress visible to the entire team.

It’s important that metrics do not become the objectives themselves, but rather serve as a way to demonstrate the true outcome was achieved. Metrics also are not a strategy, and quantitative data cannot be used alone to understand performance of teams.

When quantitative data is used properly in combination with qualitative information, strong communication, and trust, we’ve found the results can go beyond what can be achieved without metrics.

Re: GitHub Insights

#42
post #41

CEO of Code Climate here. We have a product Velocity ( https://codeclimate.com/ ) which offers what we call Engineering Intelligence. There's some great discussion about the value and appropriate use of data in software engineering, so I thought I'd chime in. What we've seen is that engineers inherently want to be productive, and are happiest when they can work friction-free. Unfortunately, it can be quite difficult…

> It’s important that metrics do not become the objectives themselves

How do you achieve that?

I've always seen the exact opposite.

It is very tempting to measure things, use more data to better understand the world, and in this case, the state of a project.

But you can't. The world is too complex, too rich, too noisy.

https://en.wikipedia.org/wiki/The_Treachery_of_Images

https://en.wikipedia.org/wiki/Map%E2%80%93territory_relation

Re: GitHub Insights

#43
post #41

CEO of Code Climate here. We have a product Velocity ( https://codeclimate.com/ ) which offers what we call Engineering Intelligence. There's some great discussion about the value and appropriate use of data in software engineering, so I thought I'd chime in. What we've seen is that engineers inherently want to be productive, and are happiest when they can work friction-free. Unfortunately, it can be quite difficult…

> It’s important that metrics do not become the objectives themselves How do you achieve that? I've always seen the exact opposite. It is very tempting to measure things, use more data to better understand the world, and in this case, the state of a project. But you can't. The world is too complex, too rich, too noisy. https://en.wikipedia.org/wiki/The_Treachery_of_Images https://en.wikipedia.org/wiki/Map%E2%80%93ter…

This HBR article "Many Strategies Fail Because They’re Not Actually Strategies", while not entirely about metrics, has some great recommendations for how leaders can avoid these pitfalls:

https://hbr.org/2017/11/many-strategies-fail-because-theyre-...

Their top recommendations are: A) Communicate the logic behind what you are trying to achieve; B) Make strategy execution a two-way process, not top-down; C) Let selection happen organically, through systems that cause strong initiatives to rise up to to the top; D) Find ways to make change the default, to help move beyond the status quo and existing habits

Re: GitHub Insights

#44

What is going on here? A lot of the comments on this thread seem to be confusing bad management with metric visibility. And it seems they would prefer to bury the metrics so they are not used to "snitch out" on people. Looks like a mix of dishonesty and toxic culture at work. We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make…

I think peoples concerns are quite valid, but a lot of the criticism comes from poorly designed tools. In full disclosure, I've been researching developer insights for quite a while and I will be releasing something in the near future that focuses on "developer first metrics". The goal is to showcase how code metrics can be used by developers to improve their day to day routine. And by doing so, make software metrics, less of a taboo.

When I took a hiatus from my startup a year and a half ago, I learned a lot about code metrics and how it is misused and why there is such a desperation for it. There is a reason why, GitPrime was acquired for about 180 million USD. I don't think it (GitPrime) serves developers well, but it does highlight how desperately management wants to know what is going on.

Since the details on "GitHub Insights" is pretty much hidden (unless you are willing to pay for it), I can't speak much for how "developer centric" it is, but if the focus for "GitHub Insights" is to appeal to developers as well, then I think it will go a long way to make software metrics a good thing for the industry as a whole.

Right now I'm keeping things under wraps, but I've attached some screenshots of what kind of metrics you can surface that can help developers. Note, the screenshots are not the final product, as the product is still being heavily refined. And what makes my solution unique is, everything can be validated. That is, if you want to have a conversation around why a piece of code (or codes) had such a high churn, you can drill down to the source code level to see how the changes were derived.

https://imgur.com/Oh0836a

https://imgur.com/QvmhFIr

https://imgur.com/5wEG6uv

https://imgur.com/ltTXhkd

As I mentioned early, I've been studying code metrics for a while and I strongly believe you will need developer buy in, if you want any chance of success with code metrics. So in my opinion, how positive/useful "GitHub Insights" is, will depend heavily on how developer friendly it is.

Re: GitHub Insights

#45

What is going on here? A lot of the comments on this thread seem to be confusing bad management with metric visibility. And it seems they would prefer to bury the metrics so they are not used to "snitch out" on people. Looks like a mix of dishonesty and toxic culture at work. We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make…

> What is going on here?

Maybe fear for even more corporate bullshit interfering with the already difficult job a developer has to do?

> confusing bad management with metric visibility

Hmm not so confusing IMO. And what is your definition of 'bad management'? Most management I've worked with as a developer have no clue about coding and the complexity we're working in. These metrics could be valuable for someone who knows more about the complexity of the underlying technology and what is needed to actually make the PR happen.

Re: GitHub Insights

#46

Earlier quoted context omitted.

Nope, fuck this. The moment a product mentions "From developer to CEO" (ie. includes the word "CEO"), this is bad news. As soon as you introduce any metrics involving "participation awards" (whether based on lines of code, commits, reviews/pull-requests, etc.), the company is doomed to fail at understanding how productivity/worth is measured in the development world. The fact they even mention "CEO" in their tagline…

Most metrics measuring code are bad. However, we have to try to come up with some. The idea that if you simply take away the metrics we will automatically improve the product is flawed. Consider rewrites that burn a year of time for dubious value. Or fancy "refactors" that are really big heavy redesigns that lock you into abstractions that don't make sense in six months. Or devs that open 2000-line after 2000-line pu…

I'm not sure. If the majority of the metrics give a false or misleading sense of reality, then maybe taking away the metrics would be a net win. It's entirely plausible that the whole system of measurement we've come up with in this domain is fundamentally flawed and has more costs than benefits (see financial system risk ratings circa 2008). The skepticism of new metrics could very well be warranted until there's a real demonstration of success and good faith in their use.

Re: GitHub Insights

#47
If they really had guts, they'd ask programmers to declare their background (years of experience, with that language, with static/dynamic typed languages, ...) and environment (private office / open floorplan / remote, music / silence / white noise, ...), and use this to settle once-and-for-all some basic questions the industry has had.

Re: GitHub Insights

#48

What is going on here? A lot of the comments on this thread seem to be confusing bad management with metric visibility. And it seems they would prefer to bury the metrics so they are not used to "snitch out" on people. Looks like a mix of dishonesty and toxic culture at work. We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make…

Data isn't insight. More metrics isn't more better. Most of these metrics are incommensurable across teams, people, projects...counterproductive to expose them. They aren't even robust to gaming.

Re: GitHub Insights

#49

What is going on here? A lot of the comments on this thread seem to be confusing bad management with metric visibility. And it seems they would prefer to bury the metrics so they are not used to "snitch out" on people. Looks like a mix of dishonesty and toxic culture at work. We basically built these metrics with Prometheus and grafana because our review queue kept growing. Now we keep everyone accountable, and make…

> and make these metrics part of our stand-up meetings

To each their own, if your team likes it, fine. If they just tolerate it because the decision came from above, yikes, what a nightmare, more management surveillance.

I would look for another job.

Re: GitHub Insights

#50

Earlier quoted context omitted.

Nope, fuck this. The moment a product mentions "From developer to CEO" (ie. includes the word "CEO"), this is bad news. As soon as you introduce any metrics involving "participation awards" (whether based on lines of code, commits, reviews/pull-requests, etc.), the company is doomed to fail at understanding how productivity/worth is measured in the development world. The fact they even mention "CEO" in their tagline…

I'm inclining on agreeing. Gaming commit counts and commit graphs on GitHub is already a thing. I've been asked by many recruiters why my commit graph on GitHub has gone stale, when I've worked at places that don't use GitHub, or if my commits are 99% of the time towards private repositories. I can already see the resumes of developers touting their statistical brilliance from metrics gained from this tool. It's not…

For the past 10 years I worked for companies with NDA and exclusivity clauses in the contracts.

I can only describe what I worked on superficially, I cannot contribute to other projects because all code I produce (even off-hours) that has my name attached to it is company property.

My GitHub and BitBucket profiles are bare, save for bug reports.

Every time I get interviewed I have to explain why my résumé is not more detailed and why I don't contribute to the open source community.

Post reply on HN