Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Wikimedia narrows down the app sendin 90M requests to a pic of flower

phabricator.wikimedia.org

Post reply on HN
Print view View original

21–30 of 105 posts

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#21

Post by qeternity »

lxgr wrote: post #5

This seems interesting from a legal point of view as well: Is an app downloading, but never displaying, creative commons content infringing on copyright (by not showing correct attribution and violating the CC terms)? Besides copyright, could this be considered theft of service?

> This seems interesting from a legal point of view as well

Is it? If you make a resource freely available to people online, and people access said resource, what's the legal ramification there? It would appear there is no malicious intent which would be necessary to make the case for abuse, and theft of service would be a stretch given that Wikimedia doesn't charge for their service.

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#22

Post by raverbashing »

segfaultbuserr wrote: post #12

> [...] We will thus hold back the banning of the url for now, awaiting for confirmation of the desired effect to reduce the potential harmful impact on the application users. Given how much "sample code" we found around the internet using that url, it might still be a good idea to merge the patch later just to prevent this from happening again. Looks like a case that the developers carelessly copied and pasted some…

On one side, StackOverflow has been a blessing.

On the other, it made "Copy Paste Programming" go to eleven. (There was even a C# example the other day that famously broke in a big project but I can't find it)

Maybe it would be a case of Stack Overflow linting examples to remove stuff like builtin urls and such.

I've seen "developers" complaining that example code with a very explicit >replace this part for your use case< complaining that the example didn't work. I guess making some things harder would just be an overall gain.

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#23

Post by oefrha »

> To narrow down the app, we decided to observe connections to the image from clients (phones) to our servers. We did this by opening the popular apps one-by-one and noting down the time. After doing this for all the apps, we then ran this query in Hive: SELECT * FROM wmf.webrequest WHERE year=2021 AND month=2 AND day=9 AND parse_media_file_url(uri_path).base_name='/wikipedia/commons/1/16/AsterNovi-belgii-flower-1mb.jpg' AND webrequest_source='upload' AND uri_host = 'upload.wikimedia.org' AND user_agent='-' AND ip=;

> We then found the specific app that was making the request by matching the time when it was opened and the time image was requested from our servers, restricting the results to the User-Agent '-' and from the IP we tested.

Unless I missed something, running mitmproxy/Charles etc. in front of the phone would have been way easier than querying the entirety of Wikimedia server logs and trying to match IP & timing windows.

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#24

Post by segfaultbuserr »

abetusk wrote: post #20

This reminds me of a Stack Overflow answer that became popular but instead of using 'example.com', they used some other random, but valid, URL that suddenly created a huge spike in traffic for the unsuspecting web page.

I'm now curious, how much traffic does example.com receive? Does it use Anycast? Does IETF publish statistics? Searched and found the answer here, no known statistics, but it's backed by a CDN.

* Ask HN: What does traffic to example.com look like?

https://news.ycombinator.com/item?id=8057442

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#25

Post by londons_explore »

clawoo wrote: post #13

> it fetches the image from Wikimedia Commons but does not display it If I were to guess, they use the picture as a connectivity/speed test. They probably figured Wikipedia has unlimited free bandwidth, so they didn't care.

The fact the picture has "1mb.jpg" in the file name suggests it is 1 megabyte and therefore a good candidate for use as a speed test.

However the image is in fact 160 kilobytes, so I suspect whatever speedtest is being done is getting the wrong results...

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#26

Post by rob74 »

MattGaiser wrote: post #3

> it is a popular chat/social media mobile app used in India Indians of Hacker News, what are the likely candidates?

Yeah, guess that the Wikimedia guys are not adepts of the "name and shame" method. Although it would be entirely justified in this case, and stop people trying to guess it and implicating the wrong app...

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#27

Post by jgrahamc »

Some time we'll get round to writing this up but there's a small customer of Cloudflare that gets a very high HTTP requests per second rate. It's a simple service (bit like a "what's my IP address" but not that) and it turns out that a quite popular hardware device hard-coded requests to this service and doesn't appear to cache the results and so it asks over and over and over again for the same information.

We've contacted the manufacturer and I think it's been patched but the life time of installed equipment is long...

Yesterday: over a billion HTTP requests...

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#28

Post by cypressious »

raverbashing wrote: post #22
segfaultbuserr wrote: post #12

> [...] We will thus hold back the banning of the url for now, awaiting for confirmation of the desired effect to reduce the potential harmful impact on the application users. Given how much "sample code" we found around the internet using that url, it might still be a good idea to merge the patch later just to prevent this from happening again. Looks like a case that the developers carelessly copied and pasted some…

On one side, StackOverflow has been a blessing. On the other, it made "Copy Paste Programming" go to eleven. (There was even a C# example the other day that famously broke in a big project but I can't find it) Maybe it would be a case of Stack Overflow linting examples to remove stuff like builtin urls and such. I've seen "developers" complaining that example code with a very explicit >replace this part for your use…

You're probably thinking about some app not being able to start when some other specific app is running because both were using a GUID copied from SO for implementing single-instance apps.

Possibly the SO question in question: https://stackoverflow.com/a/522874/615306

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#29

Post by londons_explore »

qeternity wrote: post #21
lxgr wrote: post #5

This seems interesting from a legal point of view as well: Is an app downloading, but never displaying, creative commons content infringing on copyright (by not showing correct attribution and violating the CC terms)? Besides copyright, could this be considered theft of service?

> This seems interesting from a legal point of view as well Is it? If you make a resource freely available to people online, and people access said resource, what's the legal ramification there? It would appear there is no malicious intent which would be necessary to make the case for abuse, and theft of service would be a stretch given that Wikimedia doesn't charge for their service.

I think OP is referring to the fact that your device is internally making a copy of the image during the download process, yet the creative commons license requires that copies of an image have attribution. The terms of the license are therefore likely not being met.

Very unlikely anyone will care...

Re: Wikimedia narrows down the app sendin 90M requests to a pic of flower

#30

Post by boomboomsubban »

lxgr wrote: post #5

This seems interesting from a legal point of view as well: Is an app downloading, but never displaying, creative commons content infringing on copyright (by not showing correct attribution and violating the CC terms)? Besides copyright, could this be considered theft of service?

Though it's interesting to think of the possible legal ramifications, I doubt there will be a court case. The "damages" looks like about ten terabytes of bandwidth, and lawyer fees would surpass that in days .

NTP domains have had a history of similar problems, and they seem to be resolved by apologizing, fixing the problem, and sometimes a donation.

Post reply on HN