Where’s the opt out ?
hackernews is very upfront that they do not really care about deletion requests or anything of that sort, so, the opt out is to not use hackernews.
Time to sue them to oblivion :D.
Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
111–120 of 178 posts
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#112Hello. I didn’t consent to any of my HN comments being used in this way. Please kindly remove them.
You absolutely did consent to this. https://www.ycombinator.com/legal/ See: User Content Transmitted Through the Site
That's a good point, and I think this will be my last post on this site. I never added much value anyway.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#113Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#114Earlier quoted context omitted.
I can't stand this and will actively discriminate against comments I notice in that voice. Even this one has "Not because [..], but because [..]"
I get your sentiment, though I think it's likely that people, on average, are going to organically start writing more and more like LLMs.
It's already begun.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#115Earlier quoted context omitted.
Are they paying for the repo space, I wonder?
someones paying to keep name dropping Iceberg(tm)
Weird accusation. Iceberg is an Apache project. I don’t think anyone gets paid when you use it so not sure what the benefit of shilling would be. It is just a table format that’s well suited for this purpose. I would expect any professional to make a similar recommendation.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#116My Hacker News items table in ClickHouse has 47,428,860 items, and it's 5.82 GB compressed and 18.18 GB uncompressed. What makes Parquet compression worse here, when both formats are columnar?
.. and Remove all the political shit-slop since COVID/AI and it's probably under a gig.
You could download the data and run that analysis yourself. I’d be interested to see it, especially your method of identifying “political shit-slop” and “AI” and the relationship to COVID. Sounds like an interesting project.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#117Earlier quoted context omitted.
not really since original comment completely missed it
not to be "that guy" but it is pretty explicitly laid out in the guidelines, with an example and everything
Then surely "little bit depressing this is still how we do things" is equally unwelcome
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#118Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#119Clickhouse should implement Parquet CDC to enable deduplication and faster uploads/downloads on HF
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#120Under the Known Limitations section deleted and dead are integers. They are stored as 0/1 rather than booleans. Is there a technical reason to do this? You have the type right there.
By "to do this" do you mean to not use booleans? It's because the value does not represent a binary true or false but rather a means by which the item is deleted or dead. So not only would it not make sense semantically, it would break if a third means were introduced.
Funny, because the HackerNews API [0] does return booleans for those fields. That is, a state, not a type of deletion or death.