Live data from Hacker News

4.5B Posts Scraped from TikTok

tiktok-api.seeksocial.io

41–50 of 71 posts

Re: 4.5B Posts Scraped from TikTok

#41
post #37

The AI keeps mentioning how a HTTP 200 can silently pollute your dataset. Why not just check contents of body? Usually APIs follow strict JSON contract for successful queries, alert or throw an error when that changes.

It's probably AI coded and hallucinated many things. That drumming up the importance of a minor thing is a real tell. Another hallucination - it hasn't found any video APIs (despite statements that it has and uploaded it). It has video metadata.

Re: 4.5B Posts Scraped from TikTok

#42
post #35

Earlier quoted context omitted.

There's an `is_video` column, perhaps containing a lot of 0's

More directly, there simply aren't any video files uploaded. It's just parquet files, which contain no video columns (I'm not even sure if it supports it).

Seems the expectation is the purchaser uses the info/parameters in the 289gb file to decide which videos to download, then uses the /v1/video/info endpoint to download individual videos.

Re: 4.5B Posts Scraped from TikTok

#43
post #33

Earlier quoted context omitted.

That's a blatant strawman - that's not a reasonable position at all. A reasonable person does not expect their private medical records to be accessible to you or me just because they are stored in an EHR system. They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to oth…

It's definitely not a strawman. Perhaps you meant that it's a false equivalency, but I don't think that's true either. With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.

Obviously I meant that any data a person expects to be public, like a public post on a social media site... I was not referring to data exposed in data breaches, which is a different subject entirely...

If that was not obvious to you then I apologize; though it really should have been, since you are encouraged to interact with others in good faith on HN.

And if you expect your medical records to be public...then what is the point of this discussion?

Re: 4.5B Posts Scraped from TikTok

#46
post #43

Earlier quoted context omitted.

It's definitely not a strawman. Perhaps you meant that it's a false equivalency, but I don't think that's true either. With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.

Obviously I meant that any data a person expects to be public, like a public post on a social media site... I was not referring to data exposed in data breaches, which is a different subject entirely... If that was not obvious to you then I apologize; though it really should have been, since you are encouraged to interact with others in good faith on HN. And if you expect your medical records to be public...then what…

The user only gave TikTok permission to use the videos according to the ToS. And considering the majority of TikTok users are children, I think it's hard to justify morally, even if you could make a legal case.

Re: 4.5B Posts Scraped from TikTok

#48
post #43

Earlier quoted context omitted.

Obviously I meant that any data a person expects to be public, like a public post on a social media site... I was not referring to data exposed in data breaches, which is a different subject entirely... If that was not obvious to you then I apologize; though it really should have been, since you are encouraged to interact with others in good faith on HN. And if you expect your medical records to be public...then what…

The user only gave TikTok permission to use the videos according to the ToS. And considering the majority of TikTok users are children, I think it's hard to justify morally, even if you could make a legal case.

A "data breach" refers to unauthorized access to nonpublic or protected data. Scraping content that is publicly viewable without logging in - or even with logging in, since an account is effectively disposable - is not a "data breach" (as far as any typical usage of the term goes).

In hiQ Labs v. LinkedIn, the 9th Circuit (US) ruled that scraping publicly accessible data does not violate the CFAA's "without authorization" clause (hiQ was bulk scraping public LinkedIn profile data - in violation of LinkedIn's ToS). The Supreme Court later specifically narrowed the CFAA in Van Buren v. United States saying "exceeds authorized access" applies to accessing areas of a system you aren't entitled to enter at all, not misusing access you legitimately have (regardless of ToS violations).

Other CFAA cases have ruled similarly - being legitimately granted access (i.e. signing up for an account, or browsing publicly without logging in, since the site is intended to be available to the public) and then misusing it is not "hacking".

So in the U.S., it's not a computer crime ("hacking"/"breach") to scrape data, and nobody uses the term "data breach" to refer to scraping publicly available data on a public site, except for apparently you.

Re: 4.5B Posts Scraped from TikTok

#49

Earlier quoted context omitted.

Would you say that this is immoral? Since it’s data that’s public anyways, I don’t see why putting it in a table and selling it is a bad thing.

What is "public" data? It admits to violating the ToS.

I could get the same data in principle by downloading the app and accessing the post. There’s no hacking or circumventing access controls. That’s what I meant by public data.

Re: 4.5B Posts Scraped from TikTok

#50
post #43

Earlier quoted context omitted.

Obviously I meant that any data a person expects to be public, like a public post on a social media site... I was not referring to data exposed in data breaches, which is a different subject entirely... If that was not obvious to you then I apologize; though it really should have been, since you are encouraged to interact with others in good faith on HN. And if you expect your medical records to be public...then what…

The user only gave TikTok permission to use the videos according to the ToS. And considering the majority of TikTok users are children, I think it's hard to justify morally, even if you could make a legal case.

Is it a data breach if I right click images in google search and save them to my disk?
Post reply on HN