Live data from Hacker News

LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

outerbounds.com

31–40 of 79 posts

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#31
post #9

>> Use the tool below to explore various topics and the sentiments they evoke. This is a cool phrase. It is personally important as when I was asked in a panel interview @ -- They asked "what do you think Twitter is? My response was "You're a global sentiment engine"" (There are a lot of conversations I'd love to have with the HN community with respect to our shared experiences, and weird history flipped-bits that ex…

> My response was "You're a global sentiment engine""

More like a sentiment engine for bot operators.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#33

I wonder if the dip is more about LLama3 70b training and data than a change in sentiment. The data cut-off was Dec 2023 for 70b. That looks to coincide with the reversal of the dip.

That's an interesting hypothesis but the words we use to express agreement and disagreement haven't changed much.

We don't try to retrieve articles/topics from the model, which would be affected by the cutoff, just asking it to analyze the sentiment or summarize the content provided in a prompt

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#34
I wanted to do an analysis of hacker news on another topic, but over a longer timespan.

I started to look into it, but in the little time I had to devote to the idea, I read that the Agolia API lets you look over a longer period, but that it is relatively costly.

I just want to look for all story titles from the beginning of time which match one of several simple search terms, and return submission date and title for an analysis I'd conduct in R.

Am I overthinking it and a simple Python script without an API code can do it?

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#35

I wanted to do an analysis of hacker news on another topic, but over a longer timespan. I started to look into it, but in the little time I had to devote to the idea, I read that the Agolia API lets you look over a longer period, but that it is relatively costly. I just want to look for all story titles from the beginning of time which match one of several simple search terms, and return submission date and title for…

even simpler, you can just do it in SQL

You can find all titles and dates since the beginning of HN in this public BigQuery dataset: https://console.cloud.google.com/marketplace/product/y-combi...

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#36
post #35

I wanted to do an analysis of hacker news on another topic, but over a longer timespan. I started to look into it, but in the little time I had to devote to the idea, I read that the Agolia API lets you look over a longer period, but that it is relatively costly. I just want to look for all story titles from the beginning of time which match one of several simple search terms, and return submission date and title for…

even simpler, you can just do it in SQL You can find all titles and dates since the beginning of HN in this public BigQuery dataset: https://console.cloud.google.com/marketplace/product/y-combi...

whoah. thank you dude!!

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#37
post #33

I wonder if the dip is more about LLama3 70b training and data than a change in sentiment. The data cut-off was Dec 2023 for 70b. That looks to coincide with the reversal of the dip.

That's an interesting hypothesis but the words we use to express agreement and disagreement haven't changed much. We don't try to retrieve articles/topics from the model, which would be affected by the cutoff, just asking it to analyze the sentiment or summarize the content provided in a prompt

True. It would be interesting to run these same tests on the 7B model to see if trend information changes or not. 7B had a march cutoff so if the aug-dec dip migrated to oct-march (or just disappeared) it would be strong evidence for training/data bias. If nothing else, comparing 7B to 70B would likely be interesting.

edit I realized too late I had the years off. It is pure coincidence of month, not a real data bias. Sorry! I still think it would be interesting to see a 7B comparison but that is just to see how well a small model could spot big trends compared to a bigger one.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#38
post #33

Earlier quoted context omitted.

That's an interesting hypothesis but the words we use to express agreement and disagreement haven't changed much. We don't try to retrieve articles/topics from the model, which would be affected by the cutoff, just asking it to analyze the sentiment or summarize the content provided in a prompt

True. It would be interesting to run these same tests on the 7B model to see if trend information changes or not. 7B had a march cutoff so if the aug-dec dip migrated to oct-march (or just disappeared) it would be strong evidence for training/data bias. If nothing else, comparing 7B to 70B would likely be interesting. edit I realized too late I had the years off. It is pure coincidence of month, not a real data bias.…

yep! And of course the new 3.1 model

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#39
post #7

Is this just using LLM to be cool? How does pure LLM with basic "In the scale between 0-10 ..." prompt stack up against traditional, battle-tested sentiment analysis tools? Gemini suggests NLTK and spaCy https://www.nltk.org/ https://spacy.io/

Because LLMs WILL dominate all NLP use cases, whether you like it or not.

Its like the linux of operating systems. Sure you can handwrite up some custom OS more specialized for a purpose. But its much easier to just use linux, which everyone understands on a basic level and is extremely robust, and modifying it slightly for the end goal.

And saying "Traditional sentiment analysis" tools are "Battle tested" is laughable. LLMs in the past year alone, probably has 1000x the cumulative usage of all sentiment analysis tools in history.

LLMs get 100 billion + each year in research, improvements, engineering, optimisations.

LLMs keep rapidly improving year to year in capabilities. Sonnet 3.5 already obliterates the original GPT-4 in every aspect.

LLMs keep getting cheaper year to year. Gemini flash is like 100x cheaper than the original GPT3.5.

You can onboard any person who can write python, to start using LLMs to perform language analysis in a day. Versus weeks to use these traditional tools.

Nearly all NLP tasks will be standardised to use LLMs as the baseline default tool. Sure there'll be some short term degradations in some specific aspect, but there's no stopping the tide.

By the way, traditional ML-based translation is also pretty much dead and replaced by LLMs. I've been seeing an explosion in fan-translations done by say Sonnet 3.5, the improvement in fluency and accuracy is just radical and extreme, I often don't even notice the AI-translation anymore.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#40
post #9

>> Use the tool below to explore various topics and the sentiments they evoke. This is a cool phrase. It is personally important as when I was asked in a panel interview @ -- They asked "what do you think Twitter is? My response was "You're a global sentiment engine"" (There are a lot of conversations I'd love to have with the HN community with respect to our shared experiences, and weird history flipped-bits that ex…

> My response was "You're a global sentiment engine"" More like a sentiment engine for bot operators.

[flagged]
Post reply on HN