Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

IMGZ – Paid image sharing

imgz.org

Post reply on HN
Print view View original

311–320 of 409 posts

Re: IMGZ – Paid image sharing

#311

Post by yarcob »

jrochkind1 wrote: post #265
jasode wrote: post #128

Earlier quoted context omitted.

>Yes, it's a database. That's where you're meant to put DATA. Most (all?) large-scale high-volume image storage architectures I'm aware of do not store billions of images as blobs in a database. Instead, they typically store the images as files . The database only holding metadata info and a pointer to the image filename location (or url if using AWS S3 or CDN). E.g. Facebook is one of the largest MySQL sites (if not…

I understand that postgres is pretty decent at storing large binary data though? I've been meaning to investigate trying storing images in postgres on a hobby web project where it could be convenient; I'd still want to make sure I was streaming bytes from postgres to the client, not loading the whole image into memory and only once fully loaded sending it to the client. Looking at the pg API's for my language/platfor…

> I understand that postgres is pretty decent at storing large binary data though?

Storing large files in PostgreSQL is possible, but I would advise against it.

PostgreSQL uses a technique called TOAST to store large objects. If you store something in a bytea column, it'll first be compressed (either deflate or a new algorithm that I forgot the name). Then it will be split into 8KB chunks (page size) and stored together with all the other data.

Since most image formats are already compressed, this just wastes a lot of CPU cycles.

Also, the way storage is allocated in PostgreSQL may bite you in the ass. By default, storage is not returned to the OS. If you delete all images, the data will stay on disk and marked as reusable. Only if you do a full rewrite of the table will the free space be made available again.

Also, backups will become a pain. The easiest way to backup PostgreSQL servers is to just do a pg_dump. It works great up to a few gigabytes of data. If you have images or other large objects in the database, then pg_dump will quickly become unfeasible and you will have to find another way to do backups. Not a deal breaker, but something you should be aware.

Also, for best performance, the database should be on fast storage (eg. NVME SSD), whereas the image data typically doesn't need fast storage.

## A Practical Example:

One project I work on is an image database with 50k images. The high resolution previews are around 20GB. The PostgreSQL database with all the metadata is around 20MB.

It takes 10 seconds to do a full backup of the database. Everytime I change something in the application, I first create a dump of the database (which is just a few MB compressed). Then I do my changes, and if I make a mistake I always have a backup that I could restore in 10 seconds.

If the images were stored in the PostgreSQL database, a full backup would take much longer and would require a fast internet connection. And it would be much harder to work with.

My recommendation: Use the filesystem for storing files.

Re: IMGZ – Paid image sharing

#312

Post by hardwaresofton »

GauntletWizard wrote: post #309
vyrotek wrote: post #5

I hope their architecture handles the HN front page load well. :) https://imgz.org/i9hxfgRu.png

I want to know what tool was used to create that image. It's fantastic and I have many uses for it.

Looks like https://www.cloudcraft.co/

Re: IMGZ – Paid image sharing

#313

Post by yesenadam »

I thought the comments to this were well above average on the entertaining scale. Pretty embarrassing, though, that probably the majority of comments seemed to take it seriously, even reprimanding the creator for not being serious enough, not taking his Business seriously enough. It seems to create dissonance among many that such a thing even exists. People seem deeply confused it's not full speed ahead, like every other commercial site, trying to grab money from users. We on HN constantly complain that the internet was taken over by smooth, malevolent, lying mega-corporations, with 1GB ToS's and thousands of employees..and here's something completely different and people entitledly shit on it.

It's a joke, people! From HN's own Stavros. Or, who knows what it is. Something Stavros did for fun and uses himself. A thing. Can't things just be allowed to be what they are? It's like people can't tolerate the existence of unique things, and have to see them as just really bad versions of a known category of things. "This round thing is bad because I can't force it into this square hole." And I guess when someone posts something to HN it gets judged as if submitted by the author and designed, badly, for the person now reading it. Ugly font! Bad colours! Unprofessional tone! Bad marketing! etc. ..Wasn't the internet 100% like that in the good old days?

Re: IMGZ – Paid image sharing

#314

Post by saimiam »

Imgz's pricing page made me wonder - if someone stops paying their dues for their imgz account, is it legal to auction off their photos 'as-is'?

I feel this would be similar to all those "storage wars" type shows where people buy a storage unit on auction after a preliminary look-see.

@stavros - if legal, put this in your ToS. "Clear your dues or else..."

Re: IMGZ – Paid image sharing

#315

Post by stavros »

yesenadam wrote: post #313

I thought the comments to this were well above average on the entertaining scale. Pretty embarrassing, though, that probably the majority of comments seemed to take it seriously, even reprimanding the creator for not being serious enough, not taking his Business seriously enough. It seems to create dissonance among many that such a thing even exists. People seem deeply confused it's not full speed ahead, like every o…

This is how I feel too, it was surprising (and educational) to see some commenters having trouble accepting that someone might have just made a thing for fun. I think it's the fact that you can buy it that trips them up.

I imagine Cards Against Humanity got a lot of the same comments.

Re: IMGZ – Paid image sharing

#316

Post by stavros »

saimiam wrote: post #314

Imgz's pricing page made me wonder - if someone stops paying their dues for their imgz account, is it legal to auction off their photos 'as-is'? I feel this would be similar to all those "storage wars" type shows where people buy a storage unit on auction after a preliminary look-see. @stavros - if legal, put this in your ToS. "Clear your dues or else..."

Hahah, that's a great idea! Maybe I should start auctioning off sealed accounts, and you see what photos are in there when you buy it.

Re: IMGZ – Paid image sharing

#317

Post by jventura »

nikolay wrote: post #306
stavros wrote: post #302

Earlier quoted context omitted.

> Do you want me to provide you with the email trail? I remember paying for the account. The question is whether it was a paid account at the time of the deletion . Paying for a year in 2010 obviously still makes the account unpaid in 2014. > Did you warn customers that their data (maybe a couple of megabytes) would be deleted? Yes, you get an email that this will happen. > Did you offer me "upgrade to keep your data…

Here's the email with the innocent subject of "historious hasn't seen you in a while!": Hello! We noticed that you haven't logged on historious in a while. Is there anything we can do to help? Please don't hesitate to reply to this email with anything you may need. To refresh your memory, historious is a new kind of bookmarking service which creates your very own, personalised search engine of things you like. Just a…

> It says "the data might be removed". Typically, you get multiple warnings before your data gets deleted, because people forget, and the warning is alarming.

Sorry, but it's quite explicit. Might be removed means will be removed!

> All these emails were unread - we all get a lot of spam and lots of informative emails and many of us just read the subject.

Is it stavros fault?

> people might just read the subject and not open it.

Your problem!

Re: IMGZ – Paid image sharing

#318

Post by saimiam »

stavros wrote: post #316
saimiam wrote: post #314

Imgz's pricing page made me wonder - if someone stops paying their dues for their imgz account, is it legal to auction off their photos 'as-is'? I feel this would be similar to all those "storage wars" type shows where people buy a storage unit on auction after a preliminary look-see. @stavros - if legal, put this in your ToS. "Clear your dues or else..."

Hahah, that's a great idea! Maybe I should start auctioning off sealed accounts, and you see what photos are in there when you buy it.

Love it! Maybe show a small percentage of the photos picked at random in the sealed account to bidders...

Re: IMGZ – Paid image sharing

#319

Post by corobo »

supperburg wrote: post #152
stavros wrote: post #139

Earlier quoted context omitted.

Yes! Definitely don't upload any child porn.

Yes! Definitely don’t upload any pictures of your young daughter taking a bath or of your own body if you are a teenager. Those are evil and hurt people.

If the picture of your young daughter taking a bath is in the NCMEC database you have much bigger issues than hosting images

Re: IMGZ – Paid image sharing

#320

Post by antihero »

If you don't mind using AWS you can just set up an S3 bucket and a point a CloudFront distro to it (takes 
    upload() {
      aws s3 cp ${@:2} $1 s3://$BUCKET_NAME
      echo https://$CLOUDFRONT_DOMAIN/$1 | pbcopy
    }
Post reply on HN