Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bluesky migrates to single-tenant SQLite

github.com

Post reply on HN
Print view View original

191–200 of 247 posts

Re: Bluesky migrates to single-tenant SQLite

#191

Post by winrid »

moreati wrote: post #122
winrid wrote: post #63

Why sha256 hash the user into to get a two character target directory? Wouldn't md5 be much faster and solve the same problem?

At a guess: that hash is performed relatively few times, so any performance difference is lost in the noise floor. Never having to answer "why did you use this insecure hash" or eliminating/minimising any possibility of a class of security problem is worth more.

This has nothing to do with security. It's just wasted CPU. I imagine you have to do this every time you make a query to lookup the users DB?

Security is not a concern here. It's just literally bucketing ids. Also, this is not needed with modern file systems.

Re: Bluesky migrates to single-tenant SQLite

#192

Post by BigTuna »

victorbjorklund wrote: post #127

Can someone that knows more about bluesky explain what data is stored in sqlite and not? Because i assume it isnt messages etc between users.

It's all your posts and replies as a user. While they currently host the only* PDS themselves, the end goal is for every end user to have their own PDS. Inrupt/SOLID calls this concept a "pod".

*(actually they just onboarded a second production PDS yesterday.. progress!)

Re: Bluesky migrates to single-tenant SQLite

#193

Post by winrid »

paulryanrogers wrote: post #65
winrid wrote: post #63

Why sha256 hash the user into to get a two character target directory? Wouldn't md5 be much faster and solve the same problem?

At their scale maybe they're worried about collisions? Or, like me, they're drowning in security tooling from corporate and don't want to have to carve out exceptions for md5 usage in each.

It could be they didn't want to explain the md5 usage, yeah. But that's kinda nuts if they do this every query.

Re: Bluesky migrates to single-tenant SQLite

#196

Post by fodkodrasz »

RaisingSpear wrote: post #130
paulryanrogers wrote: post #65

Earlier quoted context omitted.

At their scale maybe they're worried about collisions? Or, like me, they're drowning in security tooling from corporate and don't want to have to carve out exceptions for md5 usage in each.

> At their scale maybe they're worried about collisions? With their scheme, collisions are already guaranteed to happen if they have >256 users.

I guess parent meant abusable non-uniform distribution of collisions (they have collisions anyway as the take only the first two characters according to GP comment)

Re: Bluesky migrates to single-tenant SQLite

#197

Post by fodkodrasz »

alex_suzuki wrote: post #83
winrid wrote: post #63

Why sha256 hash the user into to get a two character target directory? Wouldn't md5 be much faster and solve the same problem?

This is probably not about collisions but about filesystem limitations (max number of files in a directory).

having a good hash uniformly distribute content helps scaling (by sharding of data)

Re: Bluesky migrates to single-tenant SQLite

#199

Post by numpad0 »

WinstonSmith84 wrote: post #144
numpad0 wrote: post #89

Earlier quoted context omitted.

e: burner email didn't work, sorry e4: check out dns on [my username].com

can't believe it's gone, too many smart people on this website lol

Not called hn for nothing! Glad I'm absolute bottomest on the floor in terms of intelligence or ability here

Re: Bluesky migrates to single-tenant SQLite

#200

Post by rchaud »

_heimdall wrote: post #180
anonyfox wrote: post #131

Slightly related: is Bluesky moderated good enough or do I get lots of rightwing and conspiracy crap like on twitter currently? I‘d really love to have some more civilized hub again that isn’t full of hate and anti-intellectualism.

Are you assuming that hate and anti-intellectualism are exclusively a rightwing thing?

On Twitter, the place that hired Tucker Carlson after Fox News dumped him? Yeah it is. No need for "both sides"-ing on this one.
Post reply on HN