On my site I don't need SEO, so I sprinkled it with invisible links leading to an endless Markov chain generated walls of garbage text. The robots seem to love it! (If you go similar route, don't forget rate limiting)
Ask HN: How do I stop companies from scraping my site?
51–60 of 62 posts
Re: Ask HN: How do I stop companies from scraping my site?
#52Earlier quoted context omitted.
These kinds of things are not settled by arbitrary judges making off the cuff decisions. They are settled by specialist technical judges with advice of dozens of specialists explaining the tech in detail. Check out Google vs Oracle the lawsuit about whether an API is copyrightable for some examples of how the rulings are very much based on deep understanding. The only downside is the process is slow and needs both a…
last I heard about this ruling was that APIs are copyrightable, which is terrible for open source reimplementations of all stuff
Alternatively they could mean "judges should stay away from X" but that is nonsensical given the courts being arbitrage...
Re: Ask HN: How do I stop companies from scraping my site?
#53Earlier quoted context omitted.
I do not trust a court to weigh in properly on the deep technical nuances involved.
we're talking about a law (copyright), who else can weight in besides the courts? or are you suggesting abolishing copyright? because I'm all for it
Or, if I came across a copy of Unreal Engine, what would stop me from using it and not paying Epic a dime?
Re: Ask HN: How do I stop companies from scraping my site?
#54Earlier quoted context omitted.
Sadly, this is not even clear cut, though of course this is a particularly egregious example. For instance depending on jurisdiction it might be illegal to sell seeds that can make plants that can produce viable seeds, unless you have a specific authorisation and depending on the species.
Legality has nothing to do with ethical. I by force could make it a law that you have to pay me to eat, most reasonable people would consider that an affront to ethical behavior.
Also, you know that morality is relative, and laws basically represent polities' morals, right ?
Re: Ask HN: How do I stop companies from scraping my site?
#55Re: Ask HN: How do I stop companies from scraping my site?
#56I sell apples at the market. How can I prevent people from buying my apples just to make pies or tarts and pretending those products are theirs? I grew the apple, I should be able to decide what people do with it. I have a sign that says the apples are only for eating, but people are ignoring it.
To me, the problem with ChatGPT is that someone builds a business around my work, and I don't get paid or credited for it.
I feel the same way about content farms paraphrasing my work with no addition of their own. I'd also take offense to someone paraphrasing my website to paying customers as a service.
Re: Ask HN: How do I stop companies from scraping my site?
#57OpenAI’s web scraping bot will respect your robots.txt. https://platform.openai.com/docs/gptbot/disallowing-gptbot
Re: Ask HN: How do I stop companies from scraping my site?
#58or
Take a screenshot and convert it to CSS.
If you need hyperlinks, use image maps on blank transparent PNG's with top z-index.
Re: Ask HN: How do I stop companies from scraping my site?
#59Earlier quoted context omitted.
we're talking about a law (copyright), who else can weight in besides the courts? or are you suggesting abolishing copyright? because I'm all for it
This is an interesting topic that comes up time and time again. I'd like to understand, if we abolish copyright and I make a computer game, pour all my money and time in it, what's to stop EvilBigPublisher Co from simply taking it, rebranding it and killing my company with their might? Or, if I came across a copy of Unreal Engine, what would stop me from using it and not paying Epic a dime?
2. Unreal lawyers costing you more in legal fees than you can afford