Improving Dropbox Performance: Retrieving Thumbnails
tech.dropbox.com
Improving Dropbox Performance: Retrieving Thumbnails
1–10 of 21 posts
Re: Improving Dropbox Performance: Retrieving Thumbnails
#2tldr: use SPDY, but until more universal support a similar custom hack is proposed
Neat hack though.. frustrating how much perf is lost due to HTTP sometimes
Re: Improving Dropbox Performance: Retrieving Thumbnails
#3I think they could still do better without using SPDY. Going to try and make time to test a different approach.
Re: Improving Dropbox Performance: Retrieving Thumbnails
#4Base64 and then gzip just to make it easier on javascript? Seems hard on dropbox servers.
Why not use a multipart binary response?
Re: Improving Dropbox Performance: Retrieving Thumbnails
#5What about a sprites-like system?
Since you pretty much know which pictures the user will request, just glue the next 100 thumbnails together into one big picture and then take it apart again on the device.
Re: Improving Dropbox Performance: Retrieving Thumbnails
#6Mostly you shouldn't have to hide backend latency by reordering, and if you don't reorder, you can just use JPEG image strips/sprites, which don't need double compression. JPEGs also stream, but in an order determined by the client, not the server.
In my experience, Dropbox is quite a bit slower (by 10-20x) than services optimized for serving images - using 1500ms to deliver a 25k file is very slow on the web, but it is very common when requesting files via the Dropbox web API. (I can only speculate about the reasons, but Amazon's s3 has big latencies too.)
Re: Improving Dropbox Performance: Retrieving Thumbnails
#7Base64 and then gzip just to make it easier on javascript? Seems hard on dropbox servers. Why not use a multipart binary response?
gzip is basically free on modern server hardware – if it makes life easier on mobile devices, this is a pretty easy tradeoff.
Re: Improving Dropbox Performance: Retrieving Thumbnails
#8Basically, they chunk thumbnails - client asks for the next 10 thumbnails in one request, server sends back 10 thumbnails in the response.
Re: Improving Dropbox Performance: Retrieving Thumbnails
#9Mostly you shouldn't have to hide backend latency by reordering, and if you don't reorder, you can just use JPEG image strips/sprites, which don't need double compression. JPEGs also stream, but in an order determined by the client, not the server. In my experience, Dropbox is quite a bit slower (by 10-20x) than services optimized for serving images - using 1500ms to deliver a 25k file is very slow on the web, but it…
We thought of spriting the thumbnails on the server, but this introduces a few problems:
- All thumbs would have to be available on the server before the sprite can be encoded, which defeats progressive rendering. (Progressively encoding JPEG might be a possibility, but this isn't something we explored.)
- The additional cost of encoding/decoding a large image on the server/client.
-- Ziga (author)
Re: Improving Dropbox Performance: Retrieving Thumbnails
#10Mostly you shouldn't have to hide backend latency by reordering, and if you don't reorder, you can just use JPEG image strips/sprites, which don't need double compression. JPEGs also stream, but in an order determined by the client, not the server. In my experience, Dropbox is quite a bit slower (by 10-20x) than services optimized for serving images - using 1500ms to deliver a 25k file is very slow on the web, but it…
We thought of spriting the thumbnails on the server, but this introduces a few problems: - All thumbs would have to be available on the server before the sprite can be encoded, which defeats progressive rendering. (Progressively encoding JPEG might be a possibility, but this isn't something we explored.) - The additional cost of encoding/decoding a large image on the server/client. -- Ziga (author)
Yes, you can stream JPEG pretty easily as long as you know the output size.
Also to combine JPEGs into a sprite, you can avoid all DCT/iDCT/color operations and use only the Huffman portion of the codec (decode, concat, encode). Since Huffman throughput is a considerable multiple of gzip (~10x?), I think you'd be quite a ways ahead, for memory and time.