Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

hackernewstrends.com

Post reply on HN
Print view View original

111–120 of 169 posts

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#111

Post by linmer »

GeoAtreides wrote: post #101
zX41ZdbW wrote: post #72

I host a publicly open database with Hacker News data at https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT... So you can create any sort of similar services in a single SQL query and an HTML page. I also hosted it as a publicly accessible data lake, which you can query from everywhere: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec... It is also updated in real-time.

oh hey, per HN terms and conditions I license my HN data only to HN. Can you please remove my data from the set? Thank you!

Wait, so I have to ask for every single person's permissions to use this data?

uhhhhhhhhhhhhhhhhhhhhhh

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#113

Post by GeoAtreides »

moralestapia wrote: post #110
GeoAtreides wrote: post #108

Earlier quoted context omitted.

>Y Combinator and its affiliated companies is zX41ZdbW either?

Oh, now I see my comment might be a bit harsh. I didn't consider you might now know about: https://github.com/hackernews/api

yes, and per HN terms and conditions only YC and YC affiliated (as you quoted) can use the api legally. I don't license my content to anyone else and so it shouldn't be use by anyone else, even if it's available on a free-for-all API (nice move HN, btw).

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#114

Post by jupr »

jupr wrote: post #23
ytkimirti wrote: post #2

Hello HN, This was a small project of mine after I've found out that I can simply the whole hackernews archive (~48GB) and play around with it. You can compare terms just like in google trends and you can also see the exact posts & comments from that time. I like that you can discover what went crazy in the timeline, they just come up as small burst of activity, it's quite fun to play around with it. https://hackerne…

Honestly the HN archive is very valuable. If you had it all on a local db with everything indexed you basically end up with a offline search engine. Where is this archive located you speak of?

found : https://huggingface.co/datasets/open-index/hacker-news

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#115

Post by moralestapia »

GeoAtreides wrote: post #113
moralestapia wrote: post #110

Earlier quoted context omitted.

Oh, now I see my comment might be a bit harsh. I didn't consider you might now know about: https://github.com/hackernews/api

yes, and per HN terms and conditions only YC and YC affiliated (as you quoted) can use the api legally. I don't license my content to anyone else and so it shouldn't be use by anyone else, even if it's available on a free-for-all API (nice move HN, btw).

https://github.com/HackerNews/API/blob/master/LICENSE

It's right there, you just have to click the link I shared ...

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#116

Post by codingdave »

moralestapia wrote: post #103
GeoAtreides wrote: post #101

Earlier quoted context omitted.

oh hey, per HN terms and conditions I license my HN data only to HN. Can you please remove my data from the set? Thank you!

By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, store, modify and otherwise use your User Content for any Y Combinator-related purpose in any form, medium or technology now known or later developed. @z…

> for any Y Combinator-related purpose

That is actually the key phrase. HN can provide the API, no problem. People can consume the API, no problem.. But I'd ask an attorney if API consumers can then re-release the data for purposes not related to YC. By my reading, they cannot.

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#117

Post by apitman »

arjie wrote: post #42

One useful feature would be to normalize by total so that I can see changes in something as opposed to just total site growth. Right now I have to chart a single generic parameter but if I pick poorly it’ll confuse the issue.

I'd love to see the opposite as well, ie how much has HN grown over time.

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#118

Post by snowwrestler »

GeoAtreides wrote: post #101
zX41ZdbW wrote: post #72

I host a publicly open database with Hacker News data at https://play.clickhouse.com/play?user=play#U0VMRUNUICogRlJPT... So you can create any sort of similar services in a single SQL query and an HTML page. I also hosted it as a publicly accessible data lake, which you can query from everywhere: https://github.com/ClickHouse/ClickHouse/issues/29693#issuec... It is also updated in real-time.

oh hey, per HN terms and conditions I license my HN data only to HN. Can you please remove my data from the set? Thank you!

Not sure if joking, but if this product is not republishing the text of your contributions (to which you hold copyright), you’re probably not going to convince a court to do anything here.

Generally speaking it is not a violation to scrape, index, and analyze web content as long as you don’t republish copyrighted content without a license, or violate access controls. For example: search engine indexes.

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#119

Post by moralestapia »

codingdave wrote: post #116
moralestapia wrote: post #103

Earlier quoted context omitted.

By uploading any User Content you hereby grant and will grant Y Combinator and its affiliated companies a nonexclusive, worldwide, royalty free, fully paid up, transferable, sublicensable, perpetual, irrevocable license to copy, display, upload, perform, distribute, store, modify and otherwise use your User Content for any Y Combinator-related purpose in any form, medium or technology now known or later developed. @z…

> for any Y Combinator-related purpose That is actually the key phrase. HN can provide the API, no problem. People can consume the API, no problem.. But I'd ask an attorney if API consumers can then re-release the data for purposes not related to YC. By my reading, they cannot.

You might want to read it again, then:

https://opensource.org/license/mit

Re: Show HN: I made Google Trends for Hacker News by indexing 18 years of comments

#120

Post by codingdave »

moralestapia wrote: post #119
codingdave wrote: post #116

Earlier quoted context omitted.

> for any Y Combinator-related purpose That is actually the key phrase. HN can provide the API, no problem. People can consume the API, no problem.. But I'd ask an attorney if API consumers can then re-release the data for purposes not related to YC. By my reading, they cannot.

You might want to read it again, then: https://opensource.org/license/mit

That is about the software, not the data.
Post reply on HN