Live data from Hacker News

Gallery-dl – download images from several image hosting sites

github.com

61–68 of 68 posts

Re: Gallery-dl – download images from several image hosting sites

#61
post #23

Earlier quoted context omitted.

ah, but i want a tarball of every photo/photogallery that i have liked . This seems to just be what I’ve ever posted.

not sure if this is sarcasm, but photos you liked aren't yours so this would be violating their privacy rules.

>privacy rules

Yet those are pictures posted on the Internet, I don't think people have a strong conviction that others can't save them

Re: Gallery-dl – download images from several image hosting sites

#62
post #57
post #52

Earlier quoted context omitted.

That's exactly what the weboob project aims to do: modularize backends (ie website) and applications so you can "use" website content the way you want https://weboob.org/

This comment comes up every time weboob is mentioned, but it'd probably see much better adoption if it was named anything other than "we boob". It's a bit of an off-putting name.

An old post about the views of the author concerning the complainers: https://laurent.bachelier.name/2013/12/weboob-the-asshole-de...

A more constructive approach to doing what every complainer asked without atually doing anything about it: https://laurent.bachelier.name/2018/07/a-proposal-for-saniti...

Re: Gallery-dl – download images from several image hosting sites

#63
post #6

Not to take anything from how great it is to have something like this, some of the supported sites have heavy rate limiting and bot detection and using this with your account can easily get you banned. For example, I had immense difficulty parsing my own saved posts from instagram (used a one-off script that runs in the browser).

I have been scraping regularly from Instagram (and a bunch of different sites) using gallery-dl for a while now, and I have yet to face any issues. Granted, I scrape at most only once a day, in a batch. So I don't know what problems you might face if you scrape more often.

The only rate-limit issues I've faced is with Twitter, because the way I do it is that I feed gallery-dl a text file containing the profile URLs that I want to scrape. But gallery-dl doesn't add a timeout in-between each input URLs, so Twitter might force a temporary cooldown on you if your list is a bit long.

But you can easily avoid this by writing your own shell script with your own custom timeout.

Re: Gallery-dl – download images from several image hosting sites

#64
post #59

Earlier quoted context omitted.

Huh, I thought it was properly FOSS albeit with a plea for subscriptions, but that's much worse. Evidently my gut instinct was right.

I think it is OSS [1], although all the code is in Subversion and I don't see a web interface. I haven't dug through it much, but poking around inside the "trunk" module, I see files with headers that say they're GPLv3. EDIT: Here's a Git mirror of trunk [2]. See, e.g., this file [3]. [1]: https://svn.jdownloader.org/projects/jd [2]: https://github.com/mirror/jdownloader [3]: https://github.com/mirror/jdownloader/blo…

> I think it is OSS [1], although all the code is in Subversion

What has the license with the chosen version control system to do?

Re: Gallery-dl – download images from several image hosting sites

#66
post #59

Earlier quoted context omitted.

I think it is OSS [1], although all the code is in Subversion and I don't see a web interface. I haven't dug through it much, but poking around inside the "trunk" module, I see files with headers that say they're GPLv3. EDIT: Here's a Git mirror of trunk [2]. See, e.g., this file [3]. [1]: https://svn.jdownloader.org/projects/jd [2]: https://github.com/mirror/jdownloader [3]: https://github.com/mirror/jdownloader/blo…

> I think it is OSS [1], although all the code is in Subversion What has the license with the chosen version control system to do?

The emphasis was more on the "no web interface" part. In 2020, I feel like Subversion is significantly less common than git, so having to install SVN + possibly learn SVN commands is a moderate barrier to browsing the code.

Re: Gallery-dl – download images from several image hosting sites

#67
post #66

Earlier quoted context omitted.

> I think it is OSS [1], although all the code is in Subversion What has the license with the chosen version control system to do?

The emphasis was more on the "no web interface" part. In 2020, I feel like Subversion is significantly less common than git, so having to install SVN + possibly learn SVN commands is a moderate barrier to browsing the code.

By the way git is an excellent SVN command-line client, though it doesn't help with the learning curve.

Re: Gallery-dl – download images from several image hosting sites

#68
post #15

Does anyone know of a downloader that works for private Facebook Groups? I'm a member of some local history groups and the material posted is amazing. I'd like to refer to it in the future, likely long after the posters are gone. I can't rely on Facebook being around then. Copying and pasting into a personal archive is slow when you want to capture everything (posts and comments) since you don't know which history yo…

Mind if I reach out to you via your contact info? I can get this into the Internet Archive.

My email address was wrong - I've fixed that now.
Post reply on HN