61–70 of 171 posts
Would be cool to see member similarity. Finding like-minded commentors/posters may help discover content that would be of interest.
Once you register on ios, you can also login through webapp: https://hn.garglet.com
probably not ready for a hacker news hug of death yet, but you can try.
Very nice. Since Hn data spawns so many such fun projects, there should be a monthly or weekly updates zip file or torrent with this data, which hackers can just download instead of writing a scraper and starting from scratch all the time.
Table definition:
CREATE TABLE hackernews_history
(
update_time DateTime DEFAULT now(),
id UInt32,
deleted UInt8,
type Enum('story' = 1, 'comment' = 2, 'poll' = 3, 'pollopt' = 4, 'job' = 5),
by LowCardinality(String),
time DateTime,
text String,
dead UInt8,
parent UInt32,
poll UInt32,
kids Array(UInt32),
url String,
score Int32,
title String,
parts Array(UInt32),
descendants Int32
)
ENGINE = MergeTree(update_time) ORDER BY id;
BATCH_SIZE=1000
TWEAKS="--optimize_trivial_insert_select 0 --http_skip_not_found_url_for_globs 1 --http_make_head_request 0 --engine_url_skip_empty_files 1 --http_max_tries 10 --max_download_threads 1 --max_threads $BATCH_SIZE"
rm -f maxitem.json
wget --no-verbose https://hacker-news.firebaseio.com/v0/maxitem.json
clickhouse-local --query "
SELECT arrayStringConcat(groupArray(number), ',') FROM numbers(1, $(cat maxitem.json))
GROUP BY number DIV ${BATCH_SIZE} ORDER BY any(number) DESC" |
while read ITEMS
do
echo $ITEMS
clickhouse-client $TWEAKS --query "
INSERT INTO hackernews_history SELECT * FROM url('https://hacker-news.firebaseio.com/v0/item/{$ITEMS}.json')"
done
“Cloud Computing” “us-east-1 down” This gave me a belly laugh.
Very nice. Since Hn data spawns so many such fun projects, there should be a monthly or weekly updates zip file or torrent with this data, which hackers can just download instead of writing a scraper and starting from scratch all the time.
It is very easy to get this dataset directly from HN API. Let me just post it here: Table definition: CREATE TABLE hackernews_history ( update_time DateTime DEFAULT now(), id UInt32, deleted UInt8, type Enum('story' = 1, 'comment' = 2, 'poll' = 3, 'pollopt' = 4, 'job' = 5), by LowCardinality(String), time DateTime, text String, dead UInt8, parent UInt32, poll UInt32, kids Array(UInt32), url String, score Int32, title…
This is impressive work, especially for a one man show! One thing that stood out to me was the graph of the sentiment analysis over time, I hadn't seen something like that before and it was interesting to see it for Rust. What were the most positive topics over time? And were there topics that saw very sudden drops? I also found this sentence interesting, as it rings true to me about social media "there seems to be a…
Thanks! Yeah I'd like to dive deeper into the sentiment aspect. As you say it'd be interesting to see some overview, instead of specific queries. The negative sentiment stood out to me mostly because I was expecting a more "clear-cut" sentiment graph: largely neutral-positive, with spikes in the positive direction around positive posts and negative around negative posts. However, for almost all my queries, the sentim…
Posts written in sweet syrupy tones wouldn’t do well here, and jokes are in short supply or outright banned. Most people here also seem to be men. There’s always someone shooting you down. And after a while, you start to shoot back.
A modern recommendation for UMAP is Parametric UMAP ( https://umap-learn.readthedocs.io/en/latest/parametric_umap.... ), which instead trains a small Keras MLP to perform the dimensionality reduction down to 2D by minimizing the UMAP loss. The advantage is that this model is small and can be saved and reused to predict on unknown new data (a traditionally trained UMAP model is large), and training is theoetically muc…
A modern recommendation for UMAP is Parametric UMAP ( https://umap-learn.readthedocs.io/en/latest/parametric_umap.... ), which instead trains a small Keras MLP to perform the dimensionality reduction down to 2D by minimizing the UMAP loss. The advantage is that this model is small and can be saved and reused to predict on unknown new data (a traditionally trained UMAP model is large), and training is theoetically muc…
This is impressive work, especially for a one man show! One thing that stood out to me was the graph of the sentiment analysis over time, I hadn't seen something like that before and it was interesting to see it for Rust. What were the most positive topics over time? And were there topics that saw very sudden drops? I also found this sentence interesting, as it rings true to me about social media "there seems to be a…
Thanks! Yeah I'd like to dive deeper into the sentiment aspect. As you say it'd be interesting to see some overview, instead of specific queries. The negative sentiment stood out to me mostly because I was expecting a more "clear-cut" sentiment graph: largely neutral-positive, with spikes in the positive direction around positive posts and negative around negative posts. However, for almost all my queries, the sentim…
It was not obvious at first glance to me, but the actual app is here: https://hn.wilsonl.in/
2) are you embedding the search phrases word by word? And using the same model as the documents used? Because I searched for „lead generation“ which any decent non-unigram embedding should understand, but I got results for lead poisoning.
https://files.casmconsulting.co.uk/message-based-community-d...
If you would like to read about my largely unsuccessful recreation of the paper, you can do so here - https://dfworks.xyz/blog/partygate/