Where’s the opt out ?
By posting comments on this site, you are relinquishing your right to that content. It belongs to YC and it is theirs to enforce, not yours. https://www.ycombinator.com/legal/
Max Schrems would like a word
Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
81–90 of 178 posts
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#82Earlier quoted context omitted.
It's not doing that. If you look at the repository, it's adding a new commit with tiny parquet files every 5 minutes. This recent one only was a 20.9 KB parquet file: https://huggingface.co/datasets/open-index/hacker-news/commi... and the ones before it were a median of 5 KB: https://huggingface.co/datasets/open-index/hacker-news/tree/... The bigger concern is how large the git history is going to get on the reposito…
I recall that this became a big problem for the Homebrew project in terms of load on the repo, to the extent that Github asked them not to recommend/default-enable shallow clones for their users: https://github.com/Homebrew/brew/issues/15497#issuecomment-1... This is likely to be lower traffic, and the history should (?) scale only linearly with new data, so likely not the worst thing. But it's something to be cogniz…
How would shallow clone be more stressful for GitHub than a regular clone?
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#83Earlier quoted context omitted.
Create a new account every so often, don’t leave any identifying information, occasionally switch up the way you spell words (British/US English), and alternate using different slang words and shorthand.
And do what I do - paste everything into ChatGPT and have it rephrase it. Not because I need help writing, but because I’d rather not have my writing style used against me.
This just gives OpenAI that data.
Perhaps you could use a local translation model to rephrase (such as TranslateGemma). If translating English to English doesn't achieve this effect then use an intermediate language, one the model is good at to not mangle meaning too much.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#84Earlier quoted context omitted.
And do what I do - paste everything into ChatGPT and have it rephrase it. Not because I need help writing, but because I’d rather not have my writing style used against me.
This just gives OpenAI that data. Perhaps you could use a local translation model to rephrase (such as TranslateGemma). If translating English to English doesn't achieve this effect then use an intermediate language, one the model is good at to not mangle meaning too much.
I run Qwen 3 locally, but I mention OpenAI on HN so people understand what I’m referring to.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#85Earlier quoted context omitted.
I recall that this became a big problem for the Homebrew project in terms of load on the repo, to the extent that Github asked them not to recommend/default-enable shallow clones for their users: https://github.com/Homebrew/brew/issues/15497#issuecomment-1... This is likely to be lower traffic, and the history should (?) scale only linearly with new data, so likely not the worst thing. But it's something to be cogniz…
How would shallow clone be more stressful for GitHub than a regular clone?
Shallow clones (and the resulting lack of shared history data) break many assumptions that packfile optimisations rely on.
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#86Where’s the opt out ?
Create a new account every so often, don’t leave any identifying information, occasionally switch up the way you spell words (British/US English), and alternate using different slang words and shorthand.
funnily enough, if everyone did this (at least make a new account often), it would prove more destructive to what HN (purposefully) wants to do than deleting the occasional account data
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#87is the legal page a placeholder, do words have no meaning? https://www.ycombinator.com/legal/ Mods, enforce your license terms, you're playing fast and loose with the law (GDPR/CPRA)
Eh, fuck that agreement. I'm kind of old school in that I believe if you put it on the internet without an auth-wall, people should be allowed to do whatever they want with it. The AI companies seem to agree. Then again, I'm not the guy that is going to get sued...
[flagged]
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#88Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#89Earlier quoted context omitted.
Create a new account every so often, don’t leave any identifying information, occasionally switch up the way you spell words (British/US English), and alternate using different slang words and shorthand.
And do what I do - paste everything into ChatGPT and have it rephrase it. Not because I need help writing, but because I’d rather not have my writing style used against me.
do the following:
sample content from users on this page: https://news.ycombinator.com/leaders
and ask the LLM to rephrase it in their voice
Re: Show HN: Hacker News archive (47M+ items, 11.6GB) as Parquet, updated every 5m
#90Hello. I didn’t consent to any of my HN comments being used in this way. Please kindly remove them.