Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

blog.wilsonl.in

Post reply on HN
Print view View original

91–100 of 171 posts

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#91

Post by darby_eight »

deadbabe wrote: post #65
wilsonzlin wrote: post #42

Earlier quoted context omitted.

Thanks! Yeah I'd like to dive deeper into the sentiment aspect. As you say it'd be interesting to see some overview, instead of specific queries. The negative sentiment stood out to me mostly because I was expecting a more "clear-cut" sentiment graph: largely neutral-positive, with spikes in the positive direction around positive posts and negative around negative posts. However, for almost all my queries, the sentim…

Anecdotally, I think anyone who reads HN for a while will realize it to be a negative, cynical place. Posts written in sweet syrupy tones wouldn’t do well here, and jokes are in short supply or outright banned. Most people here also seem to be men. There’s always someone shooting you down. And after a while, you start to shoot back.

> Anecdotally, I think anyone who reads HN for a while will realize it to be a negative, cynical place.

I don't think this is particularly unique to HN. Anonymous forums tend to attract contrarian assholes. Perhaps this place is more, erm, poorly socially-adapted to the general population, but I don't see it as very far outside the norm outside of the average wealth of the posters.

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#93

Post by dylan604 »

wilsonzlin wrote: post #42
abe94 wrote: post #33

This is impressive work, especially for a one man show! One thing that stood out to me was the graph of the sentiment analysis over time, I hadn't seen something like that before and it was interesting to see it for Rust. What were the most positive topics over time? And were there topics that saw very sudden drops? I also found this sentence interesting, as it rings true to me about social media "there seems to be a…

Thanks! Yeah I'd like to dive deeper into the sentiment aspect. As you say it'd be interesting to see some overview, instead of specific queries. The negative sentiment stood out to me mostly because I was expecting a more "clear-cut" sentiment graph: largely neutral-positive, with spikes in the positive direction around positive posts and negative around negative posts. However, for almost all my queries, the sentim…

The sentiment issue is a curious one to me. For example, a lot of humans I interact with that are not devs take my direct questioning or critical responses to be "negative" when there is no negative intent at all. Pointing out something doesn't work or anything that the dev community encounters on a daily basis isn't an immediate negative sentiment but just pointing out the issues. Is it a meme-like helicopter parent constantly doling out praise positive so that anything differing shows negativity? Not every piece of art needs to be hung on the fridge door, and providing constructive criticism for improvement is oh so often framed as negative. That does the world no favors.

Essentially, I'm not familiar with HuggingFace or any models in this regard. But if they are trained from the socials, then it seems skewed from the start to me.

Also, fully aware that this comment will probably be viewed as negative based on stated assumptions.

edit: reading further down the comments, clearly I'm not the first with these sentiments.

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#94

Post by necovek »

abe94 wrote: post #33

This is impressive work, especially for a one man show! One thing that stood out to me was the graph of the sentiment analysis over time, I hadn't seen something like that before and it was interesting to see it for Rust. What were the most positive topics over time? And were there topics that saw very sudden drops? I also found this sentence interesting, as it rings true to me about social media "there seems to be a…

It's really unfortunate the HN API does not provide votes on comments: I wonder if and how sentiment analysis would change if they were weighted by votes/downvotes?

My unsupported take is that engineers are mostly critical, but will +1 positive feedback instead of repeating it, as they might for critism :)

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#95

Post by Karrot_Kream »

beeboobaa3 wrote: post #88
Karrot_Kream wrote: post #78

Earlier quoted context omitted.

Lol what a typical comment for today's HN. Condescending ("just plain wrong") with a jab ("this isn't a hugbox") placed in just to remind you that not only are you perceived to be wrong but you've provoked anger. No proof to provoke the jab, no feedback to help fix what you perceive as wrong sentiment analysis. Just thoughtless condescension and anger. Why is the sentiment wrong? Is this a data analysis trap the OP f…

Whoosh, I was making a point by styling my comment in a way that would be perceived as negative by sentiment analysis. Good job doing a whole psychoanalysis based on what's basically a joke, though.

Heh did I miss the joke? That was a whoops indeed! Sentiment is hard on the internet ;)

> Good job doing a whole psychoanalysis based on what's basically a joke, though.

Guess there's still some work to be done on that positive sentiment replying eh? :)

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#96

Post by coolspot »

Absolutely wonderful project and even more so the writeup!

Feedback: on my iOS phone, once you select a dot on the map, there is no way to unselect it. Preview card of some articles takes full screen, so I can’t even click to another dot. Maybe add a “cross” icon for the preview card or make that when you tap outside of a card, it hides whole card strip?

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#99

Post by beeboobaa3 »

Karrot_Kream wrote: post #95
beeboobaa3 wrote: post #88

Earlier quoted context omitted.

Whoosh, I was making a point by styling my comment in a way that would be perceived as negative by sentiment analysis. Good job doing a whole psychoanalysis based on what's basically a joke, though.

Heh did I miss the joke? That was a whoops indeed! Sentiment is hard on the internet ;) > Good job doing a whole psychoanalysis based on what's basically a joke, though. Guess there's still some work to be done on that positive sentiment replying eh? :)

That one was intentional ;)

Re: Show HN: Exploring HN by mapping and analyzing 40M posts and comments for fun

#100

Post by minimaxir »

minimaxir wrote: post #73
Der_Einzige wrote: post #67

Earlier quoted context omitted.

It exists in cuML with a fast GPU implementation. Not sure why cuMl is so poorly known though…

I'll give that a look: the feature set of GPU-accelerated ops seems just up my alley for this pipeline: https://github.com/rapidsai/cuml EDIT: looking through the docs it's just GPU-acceletated UMAP, not a parametric UMAP which trains a NN model. That's easy to work around though by training a new NN model to predict the reduced dimensionality values and minimizing rMSE.

Tested it out and the UMAP implementation with this library is very very fast compared to Parametric UMAP: running it on 100k embeddings took about 7 seconds when the same pipeline on the same GPU took about a half-hour. I will definitely be playing around with it more.
Post reply on HN