Add a login with login information visible to humans. When logging in, have a prompt that says ‘ai systems are not allowed to login lest you pay $€£’
Ask HN: How do I stop companies from scraping my site?
31–40 of 62 posts
Re: Ask HN: How do I stop companies from scraping my site?
#32Steve Huffman? Is that you?
Re: Ask HN: How do I stop companies from scraping my site?
#33Earlier quoted context omitted.
Publishing a derivative work of something you scraped is not permitted. The question is whether LLMs are derivative works, which has not been settled in courts yet.
Technically correct on the point that it has not been settled in courts yet. For eg stable diffusion I actually dove a bit deeper on this: The LAION dataset is -in itself- almost indisputably legal: It contains URLs of images, and tags describing those images[1]. To be sure: _It does not contain actual images_. People have made this mistake before, and LAION quite correctly reply that they don't have them. [2] Stable…
Not sure that the size of the model is relevant.
If you take a PNG of Mickey Mouse, and create a similar vector graphic that is a fraction of the size, that would still be derivative.
Re: Ask HN: How do I stop companies from scraping my site?
#34Earlier quoted context omitted.
I think this is a poor analogy. The better comparison would be if people were buying my apples and extracting the seeds to plant their own trees, then selling their own apples at a lower price than me and putting me out of business. For example, I don't read Medium articles as often as I used to, in part because I can just ask my programming questions directly to ChatGPT. Granted, this is to be expected because ChatG…
> I think this is a poor analogy. The better comparison would be if people were buying my apples and extracting the seeds to plant their own trees, then selling their own apples at a lower price than me and putting me out of business Thereby increasing the amount of apples available to the population, lowering prices, making them available to more people. A net gain for society as a whole. This happen all the time, y…
It might also have a positive effect on artists themselves. See Jevon's Paradox on increasing efficiency of production increases demand.
Re: Ask HN: How do I stop companies from scraping my site?
#35Earlier quoted context omitted.
Publishing a derivative work of something you scraped is not permitted. The question is whether LLMs are derivative works, which has not been settled in courts yet.
Technically correct on the point that it has not been settled in courts yet. For eg stable diffusion I actually dove a bit deeper on this: The LAION dataset is -in itself- almost indisputably legal: It contains URLs of images, and tags describing those images[1]. To be sure: _It does not contain actual images_. People have made this mistake before, and LAION quite correctly reply that they don't have them. [2] Stable…
Thanks for the links! To me (merely an amateur observer), it is not perfectly obvious that the LAION dataset is legal, because I don't think "we don't have your original images anymore, we only kept the results of applying an algorithm to them" is a universal defense. If the algorithm is "resize to 640×480" you are probably still infringing, but if it's "calculate md5 checksum" you're probably not. I don't know where on the spectrum LLM training will fall, legally!
Re: Ask HN: How do I stop companies from scraping my site?
#36On my site I don't need SEO, so I sprinkled it with invisible links leading to an endless Markov chain generated walls of garbage text. The robots seem to love it!
(If you go similar route, don't forget rate limiting)
Re: Ask HN: How do I stop companies from scraping my site?
#37I sell apples at the market. How can I prevent people from buying my apples just to make pies or tarts and pretending those products are theirs? I grew the apple, I should be able to decide what people do with it. I have a sign that says the apples are only for eating, but people are ignoring it.
If we are getting philosophical, I have a spicy take of my own.
LLMs are advertised as 'language models', as if researchers coded in the Torah and the syntax of language of Eden in to a machine, and then it revealed all the truths of the world just by talking to itself. Just like AlphaZero did with chess. But in fact they are not 'language models'. They are models of what _people_ are saying. And people are, in fact, individual writers each providing their opinion and expertise. So it is a model of what Pete, Jack and Sarah said on the subject.
Except, Pete did it for add revenue, Jack to attract customers and Sarah for reputation. They won't get any of it, so they will stop writing. And so will end LLM's knowledge of novel subjects, because all it's knowledge is stolen IP. Thus, IP law is needed to protect actual producers of value.
Re: Ask HN: How do I stop companies from scraping my site?
#38Earlier quoted context omitted.
Publishing a derivative work of something you scraped is not permitted. The question is whether LLMs are derivative works, which has not been settled in courts yet.
I do not trust a court to weigh in properly on the deep technical nuances involved.
These kinds of things are not settled by arbitrary judges making off the cuff decisions.
They are settled by specialist technical judges with advice of dozens of specialists explaining the tech in detail.
Check out Google vs Oracle the lawsuit about whether an API is copyrightable for some examples of how the rulings are very much based on deep understanding.
The only downside is the process is slow and needs both a plaintiff and defendant willing to spend hundreds of thousands or millions of dollars to settle the case in court. It also takes many months of time.
Re: Ask HN: How do I stop companies from scraping my site?
#39Earlier quoted context omitted.
Publishing a derivative work of something you scraped is not permitted. The question is whether LLMs are derivative works, which has not been settled in courts yet.
I do not trust a court to weigh in properly on the deep technical nuances involved.
we're talking about a law (copyright), who else can weight in besides the courts?
or are you suggesting abolishing copyright? because I'm all for it
Re: Ask HN: How do I stop companies from scraping my site?
#40Earlier quoted context omitted.
I do not trust a court to weigh in properly on the deep technical nuances involved.
These kinds of things are not settled by arbitrary judges making off the cuff decisions. They are settled by specialist technical judges with advice of dozens of specialists explaining the tech in detail. Check out Google vs Oracle the lawsuit about whether an API is copyrightable for some examples of how the rulings are very much based on deep understanding. The only downside is the process is slow and needs both a…
last I heard about this ruling was that APIs are copyrightable, which is terrible for open source reimplementations of all stuff