Earlier quoted context omitted.
Does it make sense? Bots rarely make HTTP requests for images, css, video clips, large JS files, custom fonts, etc. Real people do. A well-written bot just seeking some specific data can often complete it's task with less than 1% of the resources that would be sent to a "real" user.
Those are all static files that are easily (and typically) cached in front of the application. Pulling customer-specific data from an authenticated session taxes the application (and DB) directly.
American Airlines and The Points Guy are suing each other
321–330 of 332 posts
Re: American Airlines and The Points Guy are suing each other
#322Earlier quoted context omitted.
Does it make sense? Bots rarely make HTTP requests for images, css, video clips, large JS files, custom fonts, etc. Real people do. A well-written bot just seeking some specific data can often complete it's task with less than 1% of the resources that would be sent to a "real" user.
Lol. Well written bot. You assume so much. Most bots are not well written at all, and more often buggy as hell generating unnecessary extra load.
Re: American Airlines and The Points Guy are suing each other
#323Earlier quoted context omitted.
It's funny how regionalism pops up on the internet. Thing is, it's less than 10% of the country (and surely not all of those 29MM are on the internet), and there are other businesses surviving just fine foregoing a TAM of the whole country.
How can you be so sure it's not regionalism that's causing you to dismiss nearly 10% of one of the largest, richest countries in the world as inconsequential?
And because HN only allows me like 3 comments a day, I'd like to emphasize "can."
Re: American Airlines and The Points Guy are suing each other
#324As an attorney who focuses in this area, I can say that the most interesting question is first where this case will be decided. Texas, and the Northern District of Texas in particular, has historically been the worst jurisdiction in the country for web scrapers to litigate. Southwest has a long history of litigating successfully there, including two cases from just last year. If TPG is going to win, first they'll nee…
As a non-american reader, it feels bizarre to me that the direction of a case changes so drastically depending on where it will be held, within the same country
Re: American Airlines and The Points Guy are suing each other
#325Earlier quoted context omitted.
So I wonder what if any legal difference there would be if TPG was only providing a web app that runs in the browser.
Also, would there be any legal difference if they wrote a browser plug-in/extension that provides additional information while you're logged into AA's site?
Re: American Airlines and The Points Guy are suing each other
#326I wrote a scraper for Air Canada's aeroplan program a few years ago. I wanted to track my points in my own custom native app. I probably had $10,000 worth of points in my account. One day I logged in to find out my account had been deactivated on suspicions of fraud. After several lengthy phone calls with their team (including sending them the node.js script I was using), I was able to get my account restored. For th…
I’ve been on the other side of this, defending against bots. Basically: Well-behaved and well-intentioned scraping bots are rare. You’d get a lot of users setting update rates to 60 seconds that did a new login every time and creating as much traffic as 1000 users. Then they’d release the script for integration with something people and suddenly you have 1000 people each creating 1000 times as many login requests as…
Re: American Airlines and The Points Guy are suing each other
#327Earlier quoted context omitted.
So, basically, you forgot to rate limit your API?
There’s no winning here. Sending back a bunch of 429’s is still part of your API. Sure it’s less expensive to do than the operation the client was probably requesting but it’s not free and it’s stateful. For the kinds of bad actors people are talking about in this thread you still want to blackhole them.
Re: American Airlines and The Points Guy are suing each other
#328Earlier quoted context omitted.
> Then of course there are people trying to make a business out of extracting your company’s data, If I can do this by hand there's no legal reason I can't do it by machine. You can try to defend against it, I guess, but the second you start impacting your obligations to someone else (like disabling their account after they paid you) you are in the wrong.
Accesses and privileges given to you does not always extend to agents acting on your behalf. Having a driver’s license does not grant someone acting on your behalf the right to drive on public roads if they don’t have their own license. And more directly it also doesn’t grant you the ability to use your autonomous driving software on public roads either.
Re: American Airlines and The Points Guy are suing each other
#329Earlier quoted context omitted.
I am not sure what "bots" you are talking about here. When I wrote a web scraper to get my Air Canada point values I used a script that fetches the web page and parses it. It was the only way i could get it to work. I had to steel the session token from the browser cookie in order to make it auth
I guarantee that, with some effort, you could write a script to emulate every HTTP call -- logging in, accepting the cookie (just a value in a Set-Cookie header), and requesting the point values, making sure that cookie value is in your Cookie header. Just because you could "only get it to work" one way, does not mean there isn't a far more efficient way.
Re: American Airlines and The Points Guy are suing each other
#330Earlier quoted context omitted.
Accesses and privileges given to you does not always extend to agents acting on your behalf. Having a driver’s license does not grant someone acting on your behalf the right to drive on public roads if they don’t have their own license. And more directly it also doesn’t grant you the ability to use your autonomous driving software on public roads either.
Except in this case the app is the car.