Bluesky migrates to single-tenant SQLite • Hacker News Board

Re: Bluesky migrates to single-tenant SQLite

#101

Post by lolinder » Tue, Nov 07, 2023, 6:49 AM UTC

krainboltgreene wrote: post #61

lolinder wrote: post #41

Earlier quoted context omitted.

I prefer to have clear commits that tell a tidy story. For example: * Refactor function `foo` to accept a second parameter * Add function `bar` * Use `bar` and `foo` in component `Baz` to implement feature #X If you give me a commit history like this, I can easily validate that each step in your claimed process does what you describe. If you instead give me a messy history and ask me to read the diff, you might know…

Funny that two of your commits don't actually tell us why they exist, one simply describes the diff (which you should never need lol?) and the other proxies that responsibility to some other system. You could have simply randomized the text in each commit, put the ticket id and the one "why" in the merge commit body and gotten the same end result amount of real information in the end.

The first line of the commit message isn't about including information that couldn't be gleaned from the commit. That can be done in subsequent lines. The first line is for two purposes:

* Priming the reader so they are able to quickly interpret what they're seeing when they open the commit.

* Making it easy to search or scan for a specific change.

The last commit message in my example would probably have included the name of the feature as well as the ticket number, but I couldn't be bothered to invent an actual feature name.

DRY doesn't really apply to technical writing, at least not as extremely as you seem to think it should. Headings are supposed to summarize the contents, and that's what commit messages are: headings.

Re: Bluesky migrates to single-tenant SQLite

#102

Post by soperj » Tue, Nov 07, 2023, 6:51 AM UTC

Tomte wrote: post #99

nyx2d wrote: post #76

Earlier quoted context omitted.

They have over 1.8 million users currently, or do you mean PDSes specifically? Federation is in open beta on a test network, you can try it out today if you'd like.

I have been on the wait list since they launched. They seem to mostly rely on invites.

It shouldn't be too difficult to find an invite? They hand them out pretty frequently.

Re: Bluesky migrates to single-tenant SQLite

#103

Post by manmal » Tue, Nov 07, 2023, 6:52 AM UTC

malkia wrote: post #16

What do they mean by "Since SQLite does not support concurrent transactions" - it supports them, as long as you don't access the .db file through a file share (UNC, or NFS, etc) - https://www.sqlite.org/wal.html I've been using this to update/read db from multiple threads/processes on the same machine. You can also do snapshotting with the sqlite backup API, if you want consistent view, and to not hold on transaction…

Did you disable auto checkpointing? Wouldn’t checkpointing result in potential corruption or at least data loss if two processes do that simultaneously? Or is that scenario exhaustively prevented with a lock file?

Re: Bluesky migrates to single-tenant SQLite

#104

Post by pmontra » Tue, Nov 07, 2023, 6:54 AM UTC

raverbashing wrote: post #79

nyx2d wrote: post #76

Earlier quoted context omitted.

They have over 1.8 million users currently, or do you mean PDSes specifically? Federation is in open beta on a test network, you can try it out today if you'd like.

Come on, "over 1.8 million users" is not an impressive number These kind of movements makes me think they're not serious about scaling up. Wouldn't surprise me if then end up as an also-ran

Maybe not impressive but none of the services of my customers had or has 1.8 million users. And yet they do well (my customers.)

Re: Bluesky migrates to single-tenant SQLite

#105

Post by pests » Tue, Nov 07, 2023, 7:01 AM UTC

defatigable wrote: post #55

JosephK wrote: post #40

Earlier quoted context omitted.

Is your offer invite code available to other randoms like myself? I tried to register on bsky months ago and still haven't been approved.

I have some extra if you'd like one. Let me know how to get it to you and I will.

I'm still trying to get one if anyone see's this. Keep missing the ones posted. Email in profile.

Thanks.

Re: Bluesky migrates to single-tenant SQLite

#106

Post by eitland » Tue, Nov 07, 2023, 7:01 AM UTC

nvy wrote: post #13

xrd wrote: post #12

Is Bluesky still invite only?

Yes. I have invite codes if you would like one. Email in my profile. Edit: they're all gone!

If anyone else still has invites I am also interested.

My mail is at the bottom of my bio.

Re: Bluesky migrates to single-tenant SQLite

#107

Post by manmal » Tue, Nov 07, 2023, 7:04 AM UTC

jakebsky wrote: post #24

Retr0id wrote: post #17

Earlier quoted context omitted.

To summarise the relevant details, the "AppView" service is responsible for the sorts of queries that aggregate across users, and that has its own database setup - I think postgres but I'm not 100% sure on that.

You're right, as usual. AppView is on a Postgres cluster with read replicas doing timeline generation (and other things) on-demand. We're in the process of moving it toward a beefy ScyllaDB cluster designed around a fanout-on-write system. The v1 backend system was optimized for rapid development and served us well. The v2 backend will be somewhat less flexible (no joins!) but is designed for much higher scale.

Does the BGS pull all the tenant‘s individual SQLite data? Or do the PDS push new posts to the BGS?

Re: Bluesky migrates to single-tenant SQLite

#108

Post by eitland » Tue, Nov 07, 2023, 7:05 AM UTC

MenhirMike wrote: post #45

lrx wrote: post #29

Earlier quoted context omitted.

I prefer to read the unified diff and commits don't matter as much.

Same. Do whatever you want in your feature branch, what matters is the Files list and the description in the PR. The whole thing gets squashed into a single commit anyway (which also makes reverting much easier).

Reverts are also easy even if one merges the whole branch. Just revert the merge commit.

I almost never look at them, but once in a while it is really great to see the thought process that led to something.

Re: Bluesky migrates to single-tenant SQLite

#109

Post by eitland » Tue, Nov 07, 2023, 7:08 AM UTC

lolinder wrote: post #51

88913527 wrote: post #48

Earlier quoted context omitted.

This is reasonable, but the problem I encounter is how stifling it seems to ask others to structure their work so specifically. By way of comparison, getting compliance on conventional commit messages is a challenge, and that's an appreciably smaller ask than this.

Oh, for sure. This is how I structure my own PRs, but I've certainly never bothered to ask a coworker to do so, I just appreciate it when I see it. That said, OP is in an environment where it sounds like this kind of structure is already the cultural norm.

From another one who tries to do the same (but doesn't enforce it):

Thanks!

Re: Bluesky migrates to single-tenant SQLite

#110

Post by numpad0 » Tue, Nov 07, 2023, 7:12 AM UTC

chocolatkey wrote: post #92

ochronus wrote: post #80

Earlier quoted context omitted.

There you go folks: bsky-social-h3d4w-u6yn4 bsky-social-74bqi-vkmcq bsky-social-n3fdq-46nxz bsky-social-yippe-32vdr bsky-social-l2fbt-xnscx

Either people were really prepared for these codes to appear, or they are being scraped. Regardless, they're all gone

It seems a temporary, anonymous, private, receive-only dropbox(not the USB drive replacement kind) on the Internet is an unsolved problem. It doesn't have to be completely out-of-band like email, could be just an encrypted public reply by `cat | base64 -d | openssl rsautl -decrypt -inkey temp.key`, so long up to few bits(70 in this instance) of encrypted content would be allowed on a platform.