Live data from Hacker News

Bluesky migrates to single-tenant SQLite

github.com

241–247 of 247 posts

Re: Bluesky migrates to single-tenant SQLite

#241
post #190

Earlier quoted context omitted.

There's nothing about security here. By this logic you should probably stop using hashmaps, then? :)

That's literally not their logic. They said: if you need security don't use md5. If you don't need security, use something faster than md5. md5 is neither secure nor fast, why use it at all?

That's dumb. Security is not a component here. They literally just want to put the files into buckets because filesystem. Also MD5 is much faster in my experience and on any benchmark I can find.

Re: Bluesky migrates to single-tenant SQLite

#242
post #191

Earlier quoted context omitted.

This has nothing to do with security. It's just wasted CPU. I imagine you have to do this every time you make a query to lookup the users DB? Security is not a concern here. It's just literally bucketing ids. Also, this is not needed with modern file systems.

all modern server CPUs have intrinsics for sha256, it just doesnt matter CPU-wise

It looks like for something the size of a UUID on Node 18, on my 8th Gen i7 (main machine broken) MD5 is only 10% faster. I guess I was remembering a time when it was like twice as fast... Neat. :)

Re: Bluesky migrates to single-tenant SQLite

#243
post #239

Earlier quoted context omitted.

It's pretty hard not to see it as a growth hack given that posts can't even be viewed without an account. That seems pretty transparently to be a system to create a feeling of FOMO/exclusivity, to make it so that you don't only need an account to participate, you need an account to even see what the network is or to follow anyone on it at all. As a comparison, Cohost limited account setup when it launched as a way to…

I asked the Bluesky devs about this back in May (of 2023). Me: "if the network is intended to be public, why are user profiles and posts currently hidden behind a login wall?" Paul Frazee: "it was a kind of bad artifact of how we set things up initially (just trying to ship). once we realized it communicated the wrong idea it was too late, and we now need to spend a heavy bit of effort communicating before we spring…

Oof. That is not a fantastic answer for them to give. I'm not even 100% sure what that means.

> Once we realized it communicated the wrong idea it was too late

For what? Too late to change the technical side of things? Is there a major technical barrier to having a public interface that matches the public firehose APIs? Because I can't figure out what that barrier would be.

What magical deadline or restriction was in place that would have prevented fixing an obvious barrier to the network?

And "once we realized it communicated the wrong idea"? People aren't misinterpreting the message, they're accurately assessing that Bluesky is not an Open network even though it is marketed as one and pretends to be one. This isn't a communication problem, it's not that blocking public access communicates the wrong idea to the public, it communicates correctly that the network isn't Open. It's complete nonsense to try and phrase a failure to fulfill the basic promises of the network as if it's actually just a PR problem.

----

> and we now need to spend a heavy bit of communicating before we spring it on everybody.

Communicating to whom? The users? Is this an admission that Bluesky users don't view the network as public or that they don't want the network to be public?

This is phrased like "we need to spend a bunch of time clarifying and explaining how this will all work before we pull the rug out from under people's feet" but who on earth would this be pulling the rug out from under? Who would be confused about this change? This isn't actually complicated; if a user without an account looks at a post it will either be visible or it won't be visible. That doesn't require a FAQ.

If the idea of that post being visible is contrary to community expectations and if the devs feel they literally can't make open decisions because the community would oppose those changes, then that's a pretty heckin big problem and it sounds like they should stop advertising that this is intended to be an open network or that federation is coming any day now, because it doesn't sound like the community is on board with that idea.

Or is it a communication problem for people outside the network? But how? What would that even mean?

Who outside of Bluesky would be confused if the devs took measures to fulfill the promises they've been publicly making since day 1 of the network? This isn't some complicated thing where people will be misinformed or they'll be confused by the idea that they can view without an account but can't post without one. That's how most networks work, locking viewing behind a login is the abnormal confusing decision to outsiders.

Ultimately, the network will be publicly viewable or it won't be. I do not understand what about that would require a PR campaign. Were people signing up for Bluesky thinking that the network was going to be permanently private? Because if so, that is something the owners should be horribly embarrassed about.

----

It's just a fundamentally weird statement. The only thing that a closed-down network communicates is that it's closed down and exclusive. The only reason that fixing the network to reflect their own marketing would be a problem is if the network doesn't want to reflect the marketing. In which case, they should stop pretending that this is a temporary limit on growth to help prevent out-of-control scaling.

The most charitable take I can have about the response is that it's corporate bullcrap from people trying to take a simple decision that was made for marketing reasons (or has accidentally been found to be extremely valuable for marketing) and to after-the-fact justify it as something complicated and difficult so that they have an excuse to avoid actually making the change.

The less charitable take I could have is that they're being honest, and they're unable to make changes to make the network more open because their userbase would be hostile to those changes or without a PR campaign would view opening up the network as if it was an attack -- and if that's the case, that sure as heck is not making me feel confident that this network has any potential at all as an Open platform. If the users aren't on board with Bluesky as a federated and Open network for everyone, then y'all don't have an Open network and it doesn't matter what your plans are.

And in either case, it's clearly not a temporary technical restriction to help with scaling so I don't know why devs are jumping into threads pretending that it is. It's clearly a deeper problem or else the devs wouldn't be giving you this kind of a nonsense response as soon as you tried to dig into it more.

Re: Bluesky migrates to single-tenant SQLite

#246
post #241

Earlier quoted context omitted.

That's literally not their logic. They said: if you need security don't use md5. If you don't need security, use something faster than md5. md5 is neither secure nor fast, why use it at all?

That's dumb. Security is not a component here. They literally just want to put the files into buckets because filesystem. Also MD5 is much faster in my experience and on any benchmark I can find.

It's really not dumb.

MD5 was designed as a cryptographic hash. It's certainly faster than many other cryptographic hashes, and it's also insecure.

Many, many hashes that are not designed to be cryptographic are 10x or more faster than md5 (e.g. murmur, siphash, xxhash, and many more).

No one thinks security is a component here. Literally everyone in this thread has agreed to that fact.

What's actually kinda dumb is to use any cryptographic hash when security is not needed - there's no need to weigh the tradeoffs between md5, sha, blake, etc. Those hashes are universally slow compared to the likes of xxhash - why not just use that to bin the files into subdirectories? Why limit yourself to the slow subset of hashes if you don't need the specific properties that make them slow?

Re: Bluesky migrates to single-tenant SQLite

#247

Earlier quoted context omitted.

I wish there were ways to enforce this on the db so you never accidentally grabbed a table lock during these operations. definitely have shot myself in the foot with postgres on this

> I wish there were ways to enforce this on the db so you never accidentally grabbed a table lock during these operations. You can use a linter for PostgreSQL migrations https://squawkhq.com/

This is awesome thank you!
Post reply on HN