Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

Post by flawsofar » Thu, May 09, 2024, 11:20 PM UTC

dylan604 wrote: post #93

wilsonzlin wrote: post #42

Earlier quoted context omitted.

Thanks! Yeah I'd like to dive deeper into the sentiment aspect. As you say it'd be interesting to see some overview, instead of specific queries. The negative sentiment stood out to me mostly because I was expecting a more "clear-cut" sentiment graph: largely neutral-positive, with spikes in the positive direction around positive posts and negative around negative posts. However, for almost all my queries, the sentim…

The sentiment issue is a curious one to me. For example, a lot of humans I interact with that are not devs take my direct questioning or critical responses to be "negative" when there is no negative intent at all. Pointing out something doesn't work or anything that the dev community encounters on a daily basis isn't an immediate negative sentiment but just pointing out the issues. Is it a meme-like helicopter parent…

Post by Karrot_Kream » Fri, May 10, 2024, 12:04 AM UTC

PaulHoule wrote: post #23

oersted wrote: post #12

This is a surprisingly big endeavour for what looks like an exploratory hobby project. Not to minimize the achievement, very cool, I'm just surprised by how much was invested into it. They used 150 GPUs and developed two custom systems (db-rpc and queued) for inter-server communication, and this was just to compute the embeddings, there's a lot of other work and computation surrounding it. I'm curious about the conte…

(1) Definitely you could use a cheaper embedding and still get pretty good results (2) I apply classical ML (say probability calibrated SVM) to embeddings like that and get good results for classification and clustering at speeds over 100x fine-tuning an LLM.

Post by chatman » Fri, May 10, 2024, 12:48 AM UTC

Post by ashu1461 » Fri, May 10, 2024, 1:34 AM UTC

wilsonzlin wrote: post #40

ashu1461 wrote: post #17

This is pretty great. Feature request : Is it possible to show in the graph how famous the topic / sub topic / article is ? So that we can do an educated exploration in the graph around what was upvoted and what was not ?

Thanks! Do you mean within the sentiment/popularity analysis graph? Or the points and topics within the map?

Post by gieksosz » Fri, May 10, 2024, 2:18 AM UTC

abe94 wrote: post #33

This is impressive work, especially for a one man show! One thing that stood out to me was the graph of the sentiment analysis over time, I hadn't seen something like that before and it was interesting to see it for Rust. What were the most positive topics over time? And were there topics that saw very sudden drops? I also found this sentence interesting, as it rings true to me about social media "there seems to be a…