Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

4.5B Posts Scraped from TikTok

tiktok-api.seeksocial.io

Post reply on HN
Print view View original

31–40 of 71 posts

Re: 4.5B Posts Scraped from TikTok

#31

Post by breezybottom »

arcfour wrote: post #27
breezybottom wrote: post #25

Earlier quoted context omitted.

What is "public" data? It admits to violating the ToS.

Something posted with the understanding that it could potentially be viewed by anyone on the Internet with minimal/no restriction. ToS are just what you follow if you don't want to get banned off of the site. If you don't care about that, then you can go hog wild, though you're being a bit of a jerk/not playing nice obviously.

Any company could potentially be hacked, so by that standard all data is public. And no, the ToS is legally binding on the company as well.

Re: 4.5B Posts Scraped from TikTok

#32

Post by igor_nast »

VulgarExigency wrote: post #19
gdevenyi wrote: post #11

Like gazing into hell

In 2026, everything we say, see or do on the internet is just another data point for some unscrupulous bastard to seek profit from.

That's not true. I build an agentic IDE - indie dev tool. And it's for free and stays like it. I do it because I enjoy coding and the technology :)

https://shikigami.dev - more details here

Re: 4.5B Posts Scraped from TikTok

#33

Post by arcfour »

breezybottom wrote: post #31
arcfour wrote: post #27

Earlier quoted context omitted.

Something posted with the understanding that it could potentially be viewed by anyone on the Internet with minimal/no restriction. ToS are just what you follow if you don't want to get banned off of the site. If you don't care about that, then you can go hog wild, though you're being a bit of a jerk/not playing nice obviously.

Any company could potentially be hacked, so by that standard all data is public. And no, the ToS is legally binding on the company as well.

That's a blatant strawman - that's not a reasonable position at all. A reasonable person does not expect their private medical records to be accessible to you or me just because they are stored in an EHR system.

They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to others.

Re: 4.5B Posts Scraped from TikTok

#35

Post by nomilk »

405error wrote: post #28
nomilk wrote: post #23

> Three things to notice, because each one bites later: Very LLMish language!

It's that mix of dense, impressive sounding jargon, but even scanning across it raises glaring problems. Like, if you have 4.5 billion videos on HF, and it's 289GB, it's about 60 bytes per video. Checking the column fields as well, there doesn't seem to be any video files*.

There's an `is_video` column, perhaps containing a lot of 0's

Re: 4.5B Posts Scraped from TikTok

#36

Post by breezybottom »

arcfour wrote: post #33
breezybottom wrote: post #31

Earlier quoted context omitted.

Any company could potentially be hacked, so by that standard all data is public. And no, the ToS is legally binding on the company as well.

That's a blatant strawman - that's not a reasonable position at all. A reasonable person does not expect their private medical records to be accessible to you or me just because they are stored in an EHR system. They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to oth…

It's definitely not a strawman. Perhaps you meant that it's a false equivalency, but I don't think that's true either.

With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.

Re: 4.5B Posts Scraped from TikTok

#37

Post by vachina »

The AI keeps mentioning how a HTTP 200 can silently pollute your dataset. Why not just check contents of body? Usually APIs follow strict JSON contract for successful queries, alert or throw an error when that changes.

Re: 4.5B Posts Scraped from TikTok

#38

Post by 405error »

nomilk wrote: post #35
405error wrote: post #28

Earlier quoted context omitted.

It's that mix of dense, impressive sounding jargon, but even scanning across it raises glaring problems. Like, if you have 4.5 billion videos on HF, and it's 289GB, it's about 60 bytes per video. Checking the column fields as well, there doesn't seem to be any video files*.

There's an `is_video` column, perhaps containing a lot of 0's

More directly, there simply aren't any video files uploaded. It's just parquet files, which contain no video columns (I'm not even sure if it supports it).

Re: 4.5B Posts Scraped from TikTok

#39

Post by simonw »

Retr0id wrote: post #7

Can any brave soul wade through the LLM prose to provide a human-readable summary?

"I uploaded 4.5 billion of those videos to Hugging Face" = I scraped the metadata (title, view count, etc) for 4.5 billion TikTok videos using the same API as their Android app. I posted the data to HF as Parquet. The video content itself is not included. I'll sell you my Go scraping code.

Re: 4.5B Posts Scraped from TikTok

#40

Post by simonw »

smallerize wrote: post #3

There's no way this dataset is going to survive on HF, right? It will be hit with so many DMCA takedowns.

It's metadata only, not video content. I expect it will likely survive - that's a pretty common pattern for machine learning datasets.

Stable Diffusion was enabled by LAION, for example. That was metadata about images and URLs to those images, but not the actual image files.

Post reply on HN