If you're not in the US, what could a company do to stop you from scraping?
block the ips, try to block the domain..lots of options
Southwest Airlines is intimidating our startup
51–60 of 76 posts
Re: Southwest Airlines is intimidating our startup
#52"The problem I see with their argument is that they are making this information public." I love this idea! However, I think that unfortunately this argument doesn't hold up, and wouldn't in court. The information is available via their site, but that does not make it public.
> The information is available via their site, but that does not make it public. Could one offer fair information if there's no reference to Southwest? Is having a website full of listings like "flightno: 123, flight_datetime: 2018-03-04-1045, price: 234.56, price_datetime: 2017-11-15-1130" something Southwest could successfully block?
If you got the info by scraping Southwest's site, I don't think it will matter if you present it without reference to Southwest. You're in trouble for how you got it, not how you present it.
Re: Southwest Airlines is intimidating our startup
#53We are giving up on free speech to avoid people been offended, of course lawyers in big companies will find a way to use that to their advantage.
Re: Southwest Airlines is intimidating our startup
#54So, the Southwest TOS state that you can't scrape or programmatically access their data. Isn't that the end of the argument unless they find a way to manually access it for customers?
Is visiting a website now some sort of contract? I have no contractual relationship with Southwest, what do I care for their TOS?
In the EU all TOS are basically just a reiteration of standard consumer law, you can put into yours whatever you want but none of that nonsense survives a legal challenge.
Re: Southwest Airlines is intimidating our startup
#55Earlier quoted context omitted.
Nice work, it even has a deploy to Heroku button! Honestly, I think it's pretty lame trying to profit from arbitrage on SW prices like that. $3 per alert? Really? Great job making an OSS implementation.
> pretty lame trying to profit from arbitrage Lame? Possibly. Profitable? Probably.
[deleted]
Re: Southwest Airlines is intimidating our startup
#56So, the Southwest TOS state that you can't scrape or programmatically access their data. Isn't that the end of the argument unless they find a way to manually access it for customers?
Is visiting a website now some sort of contract? I have no contractual relationship with Southwest, what do I care for their TOS? In the EU all TOS are basically just a reiteration of standard consumer law, you can put into yours whatever you want but none of that nonsense survives a legal challenge.
Re: Southwest Airlines is intimidating our startup
#57Earlier quoted context omitted.
Is visiting a website now some sort of contract? I have no contractual relationship with Southwest, what do I care for their TOS? In the EU all TOS are basically just a reiteration of standard consumer law, you can put into yours whatever you want but none of that nonsense survives a legal challenge.
https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
How does a TOS relate to this? Now that you've found the Wikipedia article, you might as well read it.
Re: Southwest Airlines is intimidating our startup
#58Earlier quoted context omitted.
https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
How does a TOS relate to this? Now that you've found the Wikipedia article, you might as well read it.
mapgrep has detailed that below: https://news.ycombinator.com/item?id=15715971
"I am not a lawyer and this is not legal advice, but I believe this is, sadly, dead wrong. The infamous Computer Fraud and Abuse Act contains a provision barring not just unauthorized access to computer systems but also accessing such systems in a manner that exceeds authorized access. In other words, if you break terms of service on a website, you may be in violation of the CFAA."
Re: Southwest Airlines is intimidating our startup
#59Might be worth to check out the recent LinkedIn vs. startup scraping case: https://www.reuters.com/article/us-microsoft-linkedin-ruling...
This is mis-reported. This isn't a general ruling on whether scraping sites in violation of the explicit instructions of the site owners is allowed.
The judge granted an injunction prior to the trial proper, permitting HiQ to continue scraping LinkedIn in the lead up to the trial. He did this because HiQ credibly argued that if it couldn't scrape LinkedIn it would go out of business before the court had even determined whether what it was doing was legal. These sorts of injunctions are a procedural matter and quite normal.
https://en.wikipedia.org/wiki/Preliminary_injunction has more.
Re: Southwest Airlines is intimidating our startup
#60Earlier quoted context omitted.
Set it up so the user's browser scrapes it when a button is pressed, and send a reminder every X days as chosen by the user.... User comes to your site, clicks a button, and Southwest's page loads in an iframe, where js does the lifting.
That's a neat idea. I imagine a service worker could also be used to do checks in the background, and unless SW wants to ban every IP that hits their site.. :-D
Amazon mechanical turk?
Using that you could not only have real people doing this work but also support (for low cost) people in developing nations who need money to survive.
Because of cookie policies they'd need to clear their cookies every time they query.