151–160 of 171 posts
Earlier quoted context omitted.
HN is a pretty toxic place indeed.
Perhaps... it can be toxic if you dip into the comments sometimes... Otherwise the content and links are the stuff of gold!
I set out with similar hypotheses and goals like you (on a slightly different scale though, haha) but I've been completely stuck on the interactive map part. Definitely getting a lot of pointers from how you handled this!
Maybe one key difference in approach is that I've put more emphasis on trying to extract key topics as keywords.
For ex:
article (title): "Useful Uses of cat"
keywords: ['Software design', 'Contraction', 'Code changes', 'Modularity', 'Ease of extension']
My hypothesis is this will be a faster search solution than using the embeddings, but potentially not as accurate. Not that far yet to really prove this though.
Would love to hear what you think! Any other cool ideas on what could be done with the keywords? I explain my process a bit more here if interested: https://hackernews-demo.streamlit.app/#data-aggregation-meth...
Earlier quoted context omitted.
(1) Definitely you could use a cheaper embedding and still get pretty good results (2) I apply classical ML (say probability calibrated SVM) to embeddings like that and get good results for classification and clustering at speeds over 100x fine-tuning an LLM.
I didn't think the OP used LLMs? They did use a BERT based sentiment classifier but that's not an LLM. My HN recommender works fine just using decision trees and XGBoost FWIW. I'll bet SVM would work great too.
https://huggingface.co/BAAI/bge-base-en-v1.5
is described as an "LLM" by the people who created it. It can be used in the SBERT framework.
I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with parameter tuning XGBoost would do better but it was not a winner for me.
If you look at the literature
https://arxiv.org/abs/2405.00704
you find that the fashionable LLMs are not world-beating at many tasks and actually you can do very well at sentiment analysis applying the LSTM to unpooled BERT output.
Very nice. Since Hn data spawns so many such fun projects, there should be a monthly or weekly updates zip file or torrent with this data, which hackers can just download instead of writing a scraper and starting from scratch all the time.
It is very easy to get this dataset directly from HN API. Let me just post it here: Table definition: CREATE TABLE hackernews_history ( update_time DateTime DEFAULT now(), id UInt32, deleted UInt8, type Enum('story' = 1, 'comment' = 2, 'poll' = 3, 'pollopt' = 4, 'job' = 5), by LowCardinality(String), time DateTime, text String, dead UInt8, parent UInt32, poll UInt32, kids Array(UInt32), url String, score Int32, title…
ReadWriteBufferFromHTTP: Failed to make request to 'https://hacker-news.firebaseio.com/v0/item/40298680.json'. Error: Timeout: connect timed out: 216.239.32.107:443. Failed at try 3/10. Will retry with current backoff wait is 200/10000 ms.
I googled with no luck. I was wondering if you have a solution for it.
Earlier quoted context omitted.
I didn't think the OP used LLMs? They did use a BERT based sentiment classifier but that's not an LLM. My HN recommender works fine just using decision trees and XGBoost FWIW. I'll bet SVM would work great too.
Some of the SBERT models now are based on T5 and newer architectures so there's not. The FlagEmbedding model that the author uses https://huggingface.co/BAAI/bge-base-en-v1.5 is described as an "LLM" by the people who created it. It can be used in the SBERT framework. I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with param…
Oh thanks! Right I had heard about T5 based embeddings but didn't realize it was basically an LLM.
> I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with parameter tuning XGBoost would do better but it was not a winner for me.
XGBoost worked the best for me but maybe I should retry with other techniques.
> you find that the fashionable LLMs are not world-beating at many tasks and actually you can do very well at sentiment analysis applying the LSTM to unpooled BERT output.
Definitely. Use the right tool for the right job. LLMs are probably massive overkill here. My non-LLM based embeddings work just fine for my own recommender so shrug.
Earlier quoted context omitted.
Some of the SBERT models now are based on T5 and newer architectures so there's not. The FlagEmbedding model that the author uses https://huggingface.co/BAAI/bge-base-en-v1.5 is described as an "LLM" by the people who created it. It can be used in the SBERT framework. I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with param…
> Some of the SBERT models now are based on T5 and newer architectures so there's not. The FlagEmbedding model that the author uses Oh thanks! Right I had heard about T5 based embeddings but didn't realize it was basically an LLM. > I tried quite a few models for my RSS feed recommender (applied after taking the embedding) and SVM came out ahead of everything else. Maybe with parameter tuning XGBoost would do better…
When it comes to titles I have a model that gets an AUC around 0.62 predicting if an article will get >10 votes and a much better one (AUC 0.72 or so) that predicts if an article that got > 10 votes will get a comment/vote ratio > 0.5, which is roughly the median. Both of these are bag-of-words and didn't improve when using an embedding. If I go back to that problem I'm expecting to try some kind of stacking (e.g. there are enough New York Times articles submitted to HN that I can train a model just for NYT articles.)
Also I have heard the sentiment that "BERT is not an LLM" a lot from commenters on HN a lot but every expert source I've seen seems to treat BERT as an LLM. It is in this category in Wikipedia for instance
https://en.wikipedia.org/wiki/Category:Large_language_models
and
https://www.google.com/search?client=firefox-b-1-e&q=is+bert...
gives an affirmative answer in 8 cases out of 10, one of which denies it is a language model at all on a technicality that has since been overthrown.
Earlier quoted context omitted.
It is very easy to get this dataset directly from HN API. Let me just post it here: Table definition: CREATE TABLE hackernews_history ( update_time DateTime DEFAULT now(), id UInt32, deleted UInt8, type Enum('story' = 1, 'comment' = 2, 'poll' = 3, 'pollopt' = 4, 'job' = 5), by LowCardinality(String), time DateTime, text String, dead UInt8, parent UInt32, poll UInt32, kids Array(UInt32), url String, score Int32, title…
While trying the script, I am getting the following error - ReadWriteBufferFromHTTP: Failed to make request to ' https://hacker-news.firebaseio.com/v0/item/40298680.json '. Error: Timeout: connect timed out: 216.239.32.107:443. Failed at try 3/10. Will retry with current backoff wait is 200/10000 ms. I googled with no luck. I was wondering if you have a solution for it.
Example of how it should work:
$ ch -q "SELECT * FROM url('https://hacker-news.firebaseio.com/v0/item/40298680.json')" --format Vertical
Row 1:
──────
by: octopoc
id: 40298680
parent: 40297716
text: Oops, thanks. I guess Marx was being referenced? I had thought Marx was English but apparently he was German-Jewish[1][1] https://en.wikipedia.org/wiki/Karl_Marx
time: 1715179584
type: comment
Amazing work, I'm impressed by the scope of your project! I must say though, is it jina or bge-3/flag - the embeddings (and tokenizer?) do not do a good job on tech topics. It's fine for natural words, but searching for tech concepts like "xaml", "simd", etc cause it fall back to tokenizing the inputs and tries to grab similar sounding words. Also, just some constructive feedback, if there were some way to stop it fr…