Live data from Hacker News

Show HN: Semantic clusters and embeddings for 500k Hacker News comments

app.airtrain.ai

11–20 of 54 posts

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#11

I'm not sure how to view the embedding, I clicked on the graph, narrowed down to a comment, but it only shows the row and not the raw array? (Or I've misunderstood)

Oh you cannot actually view the raw embedding vector, only the corresponding row. Any particular use-case to view the raw embedding?

None at all, pure curiosity thanks!

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#12

I did not consent on my HN data being used by entities other than HN[1]. Please remove all my comments and data from this dataset. [1] As per HN license: https://www.ycombinator.com/legal/

You did, although I have no idea if OP is an affiliate of YC (IANAL but maybe you could argue the public API is a form of sublicensing the data?): > By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, stor…

> Y Combinator and its affiliated companies

My data is licensed only to Y Combinator and its affiliated companies (I would have prefer it to be licensed it only for news.ycombinator,com), not any other rando that can access it via a browser or an API

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#13

I did not consent on my HN data being used by entities other than HN[1]. Please remove all my comments and data from this dataset. [1] As per HN license: https://www.ycombinator.com/legal/

The original dataset is located at [1] (not our HF account). HN data is directly available via the HN API [2]. The privacy policy you point to does not cover HN posts. [1] https://huggingface.co/datasets/OpenPipe/hacker-news [2] https://github.com/HackerNews/API

Well, then the original dataset should also remove my data. Just because it's there doesn't mean it's there legally.

It covers all User Content, which is comments and post titles (I think) at a minimum.

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#16
post #14

Some users comment frequently and uniquely enough that they get their own cluster. I also like how there's "Deno vs Node" lol. It'd be cool if you could click on a comment and get the most similar ones by whatever distance metric you used.

It's on our roadmap to be able to click on rows and see similar ones; thanks for the feedback!

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#18

Earlier quoted context omitted.

The original dataset is located at [1] (not our HF account). HN data is directly available via the HN API [2]. The privacy policy you point to does not cover HN posts. [1] https://huggingface.co/datasets/OpenPipe/hacker-news [2] https://github.com/HackerNews/API

Well, then the original dataset should also remove my data. Just because it's there doesn't mean it's there legally. It covers all User Content, which is comments and post titles (I think) at a minimum.

That policy explicitly excludes HN comments.

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#19

Earlier quoted context omitted.

You did, although I have no idea if OP is an affiliate of YC (IANAL but maybe you could argue the public API is a form of sublicensing the data?): > By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, stor…

> Y Combinator and its affiliated companies My data is licensed only to Y Combinator and its affiliated companies (I would have prefer it to be licensed it only for news.ycombinator,com), not any other rando that can access it via a browser or an API

Sematic (OP's employer; rebranded to Airtrain?) is a YC company. Not a lawyer but I assume that would be included in "affiliated" since YC presumably has some ownership of them.

https://www.ycombinator.com/companies/airtrain-ai

Re: Show HN: Semantic clusters and embeddings for 500k Hacker News comments

#20

I did not consent on my HN data being used by entities other than HN[1]. Please remove all my comments and data from this dataset. [1] As per HN license: https://www.ycombinator.com/legal/

That's a contract between users and HN. Airtrain is a 3rd-party.

If HN API exposes personal information publicly through their API then there is a problem.

And AFAICT the only way for HN to prevent user comments from being used by 3rd-party is preventing access to those comments, meaning a) sign-up will have to be more stringent and b) visitors will have to sign-in just to read (or scrape) comments.

Post reply on HN