Aside: That font is really hard on my eyes. Anyone else?
Me as well. Calling it painful would be overstating it, but it's not pleasant.
We sped up bun by 100x
11–20 of 68 posts
Re: We sped up bun by 100x
#12So, they implemented a git client in zig, that had some significant speedups for their usecase. However: > The git CLI test suite consists of 21,329 individual assertions for various git subcommands (that way we can be certain ziggit does suffice as a drop-in replacement for git). > While we only got through part of the overall test suite, that's still the equivalent of a month's worth of straight developer work (aga…
Reminds me when I was happy with my algorithm being super fast until I started tackling edge cases. Suddenly it's got quite slow.
Re: We sped up bun by 100x
#13[flagged]
But then there's this: "When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution."
Sooo, after burning these 10k+ worth of tokens we find out that it's sensible to use it because the language (zig) feels good as opposed to git itself which now has +20 years of human eval on it. That seems. Well. Yeah...
Re: We sped up bun by 100x
#14Seems like they actually sped bun up ~1x:
When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in all, it seems like a sensible upstream contribution.
Re: We sped up bun by 100x
#15So, they implemented a git client in zig, that had some significant speedups for their usecase. However: > The git CLI test suite consists of 21,329 individual assertions for various git subcommands (that way we can be certain ziggit does suffice as a drop-in replacement for git). > While we only got through part of the overall test suite, that's still the equivalent of a month's worth of straight developer work (aga…
Wait, so they don’t have test parity with git? How do they know that they, umm … did the actual thing they were trying to do?
Re: We sped up bun by 100x
#16[deleted]
Re: We sped up bun by 100x
#17So, they implemented a git client in zig, that had some significant speedups for their usecase. However: > The git CLI test suite consists of 21,329 individual assertions for various git subcommands (that way we can be certain ziggit does suffice as a drop-in replacement for git). > While we only got through part of the overall test suite, that's still the equivalent of a month's worth of straight developer work (aga…
> The bun team has already tested using git's C library and found it to be consistently slower hence resorting to literally executing the git CLI when performing bun install.
I find that to be a much more remarkable claim. Git doesn't have a C library, and even if it did, In which world is literally shelling out faster than a C library call? I suppose libgit2 could be implemented poorly.
If we follow their link[1] we get some clarity. It's an markdown (ai prompt?) file which just states it. Apparently they've "microbenchmarked" creating new git repositories. I really wonder if creating new git repositories is really on their hot path, but whatever.
Where does that claim in that random markdown file come from then? Well apparently, 3 years ago when somebody was "restructuring docs"[2] and just made it up.
I guess there really is a class of "engineers" that the AI can replace.
[1]: https://github.com/oven-sh/bun/blob/3ed4186bc8db8357c670307f... [2]: https://github.com/oven-sh/bun/commit/011e157cac7698050370e2...
Re: We sped up bun by 100x
#18With the recent barrage of AI-slop 'speedup' posts, the first thing I always do to see if the post is worth a read is doing a Ctrl+F "benchmark" and seeing if the benchmark makes any fucking sense.
99% of the time (such as in this article), it doesn't. What do you mean 'cloneBare + findCommit + checkout: ~10x win'? Does that mean running those commands back to back result in a 10x win over the original? Does that mean that there's a specific function that calls these 3 operations, and that's the improvement of the overall function? What's the baseline we're talking about, and is it relevant at all?
Those questions are partially answered on the much better benchmark page[1], but for some reason they're using the CLI instead of the gitlib for comparisons.
[1] https://github.com/hdresearch/ziggit/blob/5d3deb361f03d4aefe...
Re: We sped up bun by 100x
#19Seems like they actually sped bun up ~1x: When evaluating the complete bun install improvements, it came out speed-wise to about the same as the existing git usage (due to networking being the big bottleneck time-wise despite more cases being slightly faster with ziggit over multiple benchmarks). Except, it's done in 100% zig and those internal improvements pile up as projects consist of more git dependencies. All in…
[deleted]
Re: We sped up bun by 100x
#20What it cost doesn't actually say what it cost. I wonder what models they used. Napkin math of Opus for everything (probably not true) with no caching suggests $67,000. Cool article though!
They mention Anthropic, so I assumed something similar. At $5 per 5 million tokens, 13 billion would cost $65,000. However, the image in the article shows over 17 billion used, which is $85,000. That's an entry-level programmer's yearly salary. It doesn't quite pass their code tests, and it's automatic code translation, so it's going to be a pretty direct transcription. There's still probably a lot of messy code to clean up. I'm not sure it's worth it.