Seems like a hell of a lot of engineering effort for something that could easily be replaced by a gradient...
This describes at least half of all programming efforts
Lorem Picsum, death by a million pixel-gigabits
11–20 of 55 posts
Re: Lorem Picsum, death by a million pixel-gigabits
#12nice to see a digital ocean site with some popularity not getting nuked or otherwise blacklisted.
Does that really happen?
Re: Lorem Picsum, death by a million pixel-gigabits
#13I wonder how much it would cost to do a similar service and just upload all images and dimension possibilities to S3 + use cloudfront cdn.
What's the point? You'll create dimensions never used by users (CPU sensitive), you'll store files not used by users (storage), you'll miss some configurations for sure. They're creating files once and store them in a cache (CDN and local cache). Basically, they are doing something similar to your suggestion minus creating unnecessary files.
Re: Lorem Picsum, death by a million pixel-gigabits
#14Earlier quoted context omitted.
Why would it get nuked or black listed ?
DO has sometimes been accused of overselling capacity and then terminating services when they are fully used. I have never actually heard a first-hand account of this, but I assume that is what the parent is talking about.
I'm not sure if this is what you're talking about, or if the scale just wasn't a problem, but anecdotally I've (accidentally) pegged a 5$/mo instance for at least a month without any issues.
EDIT: I do know that DO/Linode/(maybe vultr) have blackholed IPs during large scale DDOS attacks, simply because they don't have the infrastructure to mitigate it, and you affect their other customers.
Re: Lorem Picsum, death by a million pixel-gigabits
#15Seems like a hell of a lot of engineering effort for something that could easily be replaced by a gradient...
It's a good exercise to test these technologies and how to run things at scale.
A real image from third party is valuable since gradients don't have onload events for example.
Re: Lorem Picsum, death by a million pixel-gigabits
#16Image resizer scaling is one of the more interesting problems I have worked on in the last 10 years of so. I was part of a small team that designed and built the resizer that powers the nine.com.au network of sites. Modest by USA standards it gets close to hundreds of millions of views a day across the whole network.
We ended up using shared nothing architecture. The whole thing ran on 6 T2 large AWS instances using a slightly modified version of Thumbor where if you rotated the key the disk cache would be shared so avoid a large scale cache invalidation. It worked quite well and we rotated the key every few weeks.
Things I learnt.
Pretty much all image resizers have the same performance as all the good ones call out to C libraries in the end. Akamai (the CDN we used) despite having site-shield on would still hit the back-end ~100 times for the same image on occasion as I suspect all of the whitelisted machines could request the same image if their internal sharing didn't kick in fast enough.
Long tail images were the ones that brought the resizer to its knees. The hot images would quickly enter the local disk cache and were not an issue. Purge the whole cache though and the long tail images would quickly overwhelm the instances.
The last thing I learnt was to have a backup cloud-front ready to flip over to. At one point Akamai had issues and the resizer was facing origin load. It capped out at about 300 RPS which couldn't keep up with what was expected. It got even worse when the T2 instances ran out of credit. Spinning up cloud-front solved that issue once the DNS flip kicked in.
One good thing to come out of it was I helped write the C# thumbor library as we had one site that was using C# and nobody could move over to the new resizer without it.
Re: Lorem Picsum, death by a million pixel-gigabits
#17I wonder how much it costs them to maintain this service.
I run https://dummyimage.com, the first placeholder image service which has been online since 2007. I use a 1GB, 1vCPU instance from DreamCompute costing $6.00 per month. That's it.
To be fair dummyimage.com doesn't do any sort of image reading or resizing. Source doe at https://github.com/kingkool68/dummyimage
Re: Lorem Picsum, death by a million pixel-gigabits
#18I wonder how much it would cost to do a similar service and just upload all images and dimension possibilities to S3 + use cloudfront cdn.
Or just use Amazons serverless image handler, to server up requested sizes, and cache them. https://aws.amazon.com/solutions/serverless-image-handler/
Re: Lorem Picsum, death by a million pixel-gigabits
#19Seems like a hell of a lot of engineering effort for something that could easily be replaced by a gradient...
It's a good exercise to test these technologies and how to run things at scale. A real image from third party is valuable since gradients don't have onload events for example.
An image gradient then. 1 px wide perhaps, or high. Few problems resizing it (even on client) and it should be reusable.
Re: Lorem Picsum, death by a million pixel-gigabits
#20I wonder how much it would cost to do a similar service and just upload all images and dimension possibilities to S3 + use cloudfront cdn.
Well let's limit the possibilities to any image between 1x1 and 1920x1080.
Calculating the number of possible images is simple enough. It's just 1920*1080, or 2073600 images.
Now what's the average size of each image? Well the average of each dimension is half of the full size, so the average area should be 1/4th to of a 1920x1080 image, or 518400 pixels per image.
So in total, we need to save 1074954240000 pixels. Now how many bytes does each pixel take to store? I really don't know. You can save them as 8bit RGB PNG images and assume each pixel will use 24 bytes. You'd add some for headers, and remove some to account for compression, but let's ignore that for now. Maybe someone else can chime in. But for now we need to store 25798901760000 bytes, or about 23TB.
Not impossible, but sounds needlessly expensive.