111–120 of 403 posts
So, if star to fork ratio is the new signal, time to make an extra fake star tier, where the bot forks the repo, generates a commit with the cheapest LLM available and pushes that to gh, right?
We figured out a workaround to limit activity to prior contributors only, and add a CI job that pushes a coauthored commit after passing captcha on our website. It cut the AI slop by 90%. Full write-up https://archestra.ai/blog/only-responsible-ai
> When nobody is forking a 157,000-star repository, nobody is using it
that is completely not true, i don't fork a repo when i use it, only when i want to contribute to it (and usually cleanup my forks)
It’s more expensive to compute, but the resulting scores would be more trustworthy unless I’m missing something.
I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would. Here are the things I look at in order: * last commit date. Newer is better * age. old is best if still updating. New is not great but tolerable if commits aren't rapid * issues. Not the count, mind you, just looking at them. How are they handled, what kind of issues are lingering open. * some of the co…
> I don't think I have ever used stars in making a decision to use a library and I don't understand why anyone would. You might not have but the makers of dependencies that you use might so still problematic.
I have limited time on this Earth and at my employer. My job is not critical to life. I am comfortable with this level of pragmatism.
These kinds of articles make you feel like there are specific, actionable problems that just need an adjustment and then they disappear. However, the system is much worse than you'd expect. Studies like this are extremely valuable, but they don't address the systematic problems affecting all signaling channels: most signals themselves have been manufactured into a product. Build a SaaS and you'll have "journalists" a…
It all boils down to making more money.
I'm increasingly convinced the issue isn't feedback itself, but centralized, global, aggregated feedback that becomes game-able without stronger identity signals.
Right now the incentives are tied (correctly or not) to these global metrics, so you get a market for faking them, with money flowing to whoever is best at juicing that signal.
If instead the signal was based on actual usage and attributions by actual developers, the incentives shift. With localized insight (think "Yeah, I like Golang") it becomes both harder to fake and harder to get at the metric rollup.
Useful reputation on the web is actually much more localized and personal. I gladly receive updates on and would support the repos I've starred. If I could chose where to put my dollars (not an investors), it would likely include the list of repos I've personally curated.
This suggests a different direction: instead of asking "how many stars does this have?", ask "who is actually depending on this, and in what context?" or better retroactively compare your top-n repos to mine and we'll get a metric seen through our lenses. If you want to include everyone in that aggregation you'll end up where we are now, but if in stead you chose the list, well, the stars could align as a good metric once more.
The interesting part is that the web already contains most of that information, we just don't treat identity as a part of the signal (yet? universally?).
We should do a hall of shame!
I wonder if it makes sense for GitHub to use graph-theoretic measures like PageRank instead of raw stars. In simple terms, a repo is considered important if it is starred or forked by GitHub users who maintain other important repos. It’s more expensive to compute, but the resulting scores would be more trustworthy unless I’m missing something.
I run a tiny site that basically gave a point-at-able definition to an existing adhoc standard. As part of the effort I have a list of software and libraries following the standard on the homepage. Initially I would accept just about anything but as the list grew I started wanting to set a sort of notability baseline. Specifically someone submitted a library that was only several days old, clearly entirely AI generat…
> When a metric becomes a target it ceases to be a good metric as they say. https://en.wikipedia.org/wiki/Goodhart's_law
I've been thinking about this a lot. These metrics are all just marketing signals to draw people's attention, trying to make some kind of deals. So the fix should be: make the cost of the signal match what it claims to represent. I'm obsessed with something called DUKI /djuːki/ (Decentralized Universal Kindness Income, a form of UBI) — the idea is that instead of stars or reviews, trust comes from deals pledging real money to the world for all as the deal happens. You can't fake that cheaply.
So the metric becomes the money itself — if you fake X amount, it costs you X, and the world will thank you by paying attention...
Imagine if GitHub let you back a star with real money — the more you put in, the more credible the star. And that money goes out as UBI for everyone. For attention makers, star anything you want, as much as you want. For attention takers, just follow the money to filter through all the noise that's so easy to manipulate...