Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

outerbounds.com

Post reply on HN
Print view View original

61–70 of 79 posts

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#61

Post by JimDabell »

t-writescode wrote: post #56
JimDabell wrote: post #55

Earlier quoted context omitted.

What do you mean? Deploying something like spaCy is far easier than deploying an LLM in my experience.

By "deploy", they almost certainly mean "set up to use" and they may have also included "learn how to use" and all its various forms, as well. LLMs really are almost magic in how they can help in this space; and setting them up is often just getting an API key and throwing some money and webservice calls at them.

Set up to use, sure. Learn? Learning isn’t deployment in this context.

Setting up spaCy is just `pip install spacy`. No need to worry about GPUs or dedicated services like you do with LLMs.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#62

Post by throwawaymaths »

JimDabell wrote: post #55
throwawaymaths wrote: post #25

Earlier quoted context omitted.

And yet, it's so much easier to deploy an LLM, either through a service or on prem.

What do you mean? Deploying something like spaCy is far easier than deploying an LLM in my experience.

pip install vllm, boom you have an openapi-compatible webserver. No further action necessary.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#63

Post by visarga »

I did a similar kind of process for my own chat logs. I have about 11M tokens worth of logs, and it took 2 days to crunch all of them with ollama and LLaMA 3.1 8B on my MacBook. It's slow, but free.

I generated title, summary, keywords and hierarchical topics up to 3 levels up from the original text. My plan for now is to put them in a vector search engine, which, incidentally, was made with Sonnet 3.5 with very little iteration. I want to play around to see how I can organize my ideas with LLMs, make something useful from all that text.

I really don't know what I will discover. One small insight I already found is that summarization works really well, you can use summaries instead of full texts to prime Claude and it works better than expected. Unlimited context? Maybe.

Another direction of research is to create a nice taxonomy, there are thousands of topics, pretty difficult task, but there must be a way using clustering and LLMs. That is why I generated topic, parent-topic, gp-topic, and ggp-topic from all snippets. I would probably manually edit the top 2 levels of the taxonomy to give it the right focus.

I'm also integrating with my HN and reddit feeds. X is too stingy with the API. Maybe Pocket and local downloads folder too, I save/bookmark stuff I like. I could also include all the papers I am reading into the corpus. It could synthesize a ranked feed aligned to my own interests.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#64

Post by throwawaymaths »

JimDabell wrote: post #61
t-writescode wrote: post #56

Earlier quoted context omitted.

By "deploy", they almost certainly mean "set up to use" and they may have also included "learn how to use" and all its various forms, as well. LLMs really are almost magic in how they can help in this space; and setting them up is often just getting an API key and throwing some money and webservice calls at them.

Set up to use, sure. Learn? Learning isn’t deployment in this context. Setting up spaCy is just `pip install spacy`. No need to worry about GPUs or dedicated services like you do with LLMs.

yeah, then you have to learn the API. Basically every option for running LLMs have converged on the openAI (web) API.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#65

Post by RamblingCTO »

anonylizard wrote: post #39
tantalor wrote: post #7

Is this just using LLM to be cool? How does pure LLM with basic "In the scale between 0-10 ..." prompt stack up against traditional, battle-tested sentiment analysis tools? Gemini suggests NLTK and spaCy https://www.nltk.org/ https://spacy.io/

Because LLMs WILL dominate all NLP use cases, whether you like it or not. Its like the linux of operating systems. Sure you can handwrite up some custom OS more specialized for a purpose. But its much easier to just use linux, which everyone understands on a basic level and is extremely robust, and modifying it slightly for the end goal. And saying "Traditional sentiment analysis" tools are "Battle tested" is laughab…

Sorry, but not really. If you know what you do, you don't just pick an LLM. LLMs are trained/built for a specific task: text generation. Other models are trained on different tasks. If you know what you do, you compare models (I don't mean LLM models with that!) and choose the best performing. Just because LLMs receive more training doesn't mean they have a better performance. Very weird and flawed way of thinking. This is just hype thinking

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#66

Post by imdsm »

tantalor wrote: post #27
throwawaymaths wrote: post #25

Earlier quoted context omitted.

And yet, it's so much easier to deploy an LLM, either through a service or on prem.

It's easier to do a lot of things. That doesn't make it better.

Sometimes, doing something different does result in something better.

For example, EVs. Compare EVs to ICEVs and you can point out a lot of faults, but ICEVs have had 100 years of refinement. Perhaps you're comparing battle hardened SA with fledgling LLM based SA?

Never don't do new things, not only when its for fun, but especially when its for fun. If you want to be closed minded, that's your choice, but don't try to put that mentality onto others.

Keep your non-hacker mindset to yourself.

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#67

Post by mithametacs »

visarga wrote: post #63

I did a similar kind of process for my own chat logs. I have about 11M tokens worth of logs, and it took 2 days to crunch all of them with ollama and LLaMA 3.1 8B on my MacBook. It's slow, but free. I generated title, summary, keywords and hierarchical topics up to 3 levels up from the original text. My plan for now is to put them in a vector search engine, which, incidentally, was made with Sonnet 3.5 with very litt…

LLMs are shit at generating content, but summarization works really well.

I’d like to use your project

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#68

Post by [deleted] »

imdsm wrote: post #66
tantalor wrote: post #27

Earlier quoted context omitted.

It's easier to do a lot of things. That doesn't make it better.

Sometimes, doing something different does result in something better. For example, EVs. Compare EVs to ICEVs and you can point out a lot of faults, but ICEVs have had 100 years of refinement. Perhaps you're comparing battle hardened SA with fledgling LLM based SA? Never don't do new things, not only when its for fun, but especially when its for fun. If you want to be closed minded, that's your choice, but don't try t…

[deleted]

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#69

Post by hitradostava »

RamblingCTO wrote: post #65
anonylizard wrote: post #39

Earlier quoted context omitted.

Because LLMs WILL dominate all NLP use cases, whether you like it or not. Its like the linux of operating systems. Sure you can handwrite up some custom OS more specialized for a purpose. But its much easier to just use linux, which everyone understands on a basic level and is extremely robust, and modifying it slightly for the end goal. And saying "Traditional sentiment analysis" tools are "Battle tested" is laughab…

Sorry, but not really. If you know what you do, you don't just pick an LLM. LLMs are trained/built for a specific task: text generation. Other models are trained on different tasks. If you know what you do, you compare models (I don't mean LLM models with that!) and choose the best performing. Just because LLMs receive more training doesn't mean they have a better performance. Very weird and flawed way of thinking. T…

I have to agree with the parent. LLMs are excellent at a large range of NLP tasks. Of course they are not going to replace all ML models, but when it comes to NLP they are clearly better than lots of trained models (e.g. https://arxiv.org/pdf/2310.18025).

Re: LLM-based sentiment analysis of Hacker News posts between Jan 2020 and June 2023

#70

Post by teo_zero »

Sleaker wrote: post #22

Why is everything only plotted between 4 and 8 if the scale of the least liked topic should be 0 and most liked should be 9. Also 4.5 is the midpoint, but 4 is displayed as bright red and 6 is a muted gray blue, why? This makes no sense except to be psychologically disingenuous. And no 5s? What is even going on in that LLM?

The scale makes no sense.

Sentiment of forum posts is not an absolute value, you can't compare it against, for example, conversations in a pub, or talks between friends, etc.

I think they should have normalized the numbers around the average, so to have a relative measurement of the various topics.

Post reply on HN