41–50 of 57 posts
I'm very curious as to how this works in the backend. I realize it uses Bluesky's firehose to get the posts, but I'm more curious on how it's checking whether a post contains any of the available words. Any guesses?
Maybe I'm being naive, but with only ~275k words to check against, this doesn't seem like a particularly hard problem. Ingest post, split by words, check each word via some db, hashmap, etc... and update metadata.
I'm very curious as to how this works in the backend. I realize it uses Bluesky's firehose to get the posts, but I'm more curious on how it's checking whether a post contains any of the available words. Any guesses?
Hey! this is my site - it's not all that complex, i'm just using a sqlite db with two tables - one for stats, the other for all the words that's just word | count | first use | last use | post. I... did not expect this to be so popular
And what ingress bandwidth do you have?
Earlier quoted context omitted.
Hey! this is my site - it's not all that complex, i'm just using a sqlite db with two tables - one for stats, the other for all the words that's just word | count | first use | last use | post. I... did not expect this to be so popular
What is your source dictionary to compare to? Seems kind of small. Also, how are you handling inflected forms?
using this, a combo of "covered enough" for the bit and easy to use
also, since i'm tracking every word (technically a better name for this project would be The Bluesky Corpus) all inflected forms are different words, which aligns with my thinking
Earlier quoted context omitted.
Hey! this is my site - it's not all that complex, i'm just using a sqlite db with two tables - one for stats, the other for all the words that's just word | count | first use | last use | post. I... did not expect this to be so popular
What are the table sizes? And what ingress bandwidth do you have?
Ingress is actually pretty manageable, ~900kbps
I'm surprised at how normal some of the unseen words are. I expected them to all be archaic or niche, but many are pretty reasonable: 'congregant', 'definer', 'stereoscope'.
For a moment I thought it would be an AT-Proto based Urban Dictionary clone.
I'm surprised at how normal some of the unseen words are. I expected them to all be archaic or niche, but many are pretty reasonable: 'congregant', 'definer', 'stereoscope'.
For what it's worth, there's 1.7bn posts on Bluesky according to this: https://bsky.jazco.dev/stats The dictionary site has only checked 4,920,000 posts, which is 0.28% of all messages.
This website is so pretty!