How does this compare to things like COGs (Cloud Optimised GeoTIFFs) or other binary blob + index raster pyramid formats?
Was there a requirement to work with these formats directly without converting?
Show HN: Streaming gigabyte medical images from S3 without downloading them
11–20 of 51 posts
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#12Please don’t use AWS S3 there’s vast numbers of much cheaper compatible choices.
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#13Please don’t use AWS S3 there’s vast numbers of much cheaper compatible choices.
Pretty bold half claim while not backing it up with a single data point. :D
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#14https://dicom.nema.org/dicom/dicomwsi/ Interesting guide to the Whole Slide Images (WSI) format. The surprising thing for me is that compression is used, and they note does not affect use in diagnostics. Back in the day we used TIFF for a similar application (X-ray detector images).
Digital pathology are just a lot bigger than radiology, we regularly see slides 500k x 500k pixels.
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#15You could probably do it completely clientside. I have a parser for 12 scanner formats in js. It doesnt read the pixels, just parses metadata but jpeg is easy and most common anyway
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#16How does this compare to things like COGs (Cloud Optimised GeoTIFFs) or other binary blob + index raster pyramid formats? Was there a requirement to work with these formats directly without converting?
Yes there is a requirement to work with the vendor format. For instance, TCGA (The Cancer Genome Atlas - a large dataset of 12k+ human tumor cases) has mostly .svs files (scanned with an Aperio scanner). We tend to work with these formats as they contain all the metadata we need.
Sometimes, it happens that we re-write the image in a pyramidal TIFF format (happened to me a few times, where NDPI images had only the highest resolution level, no pyramid), in which case COGs could work.
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#17Please don’t use AWS S3 there’s vast numbers of much cheaper compatible choices.
As data scientists, we usually don't get to choose. It's usually up to the hospital or digital lab's CISO to decide where the digitized slides are stored, and S3 is a fairly common option.
That being said, I plan to support more cloud platforms in the future, starting with GCP.
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#18How does this handle images with different compression formats?
Currently we only support TIFF and SVS with JPEG and JPEG2000 compression formats. I plan on supporting more file extensions (e.g. NDPI, MRXS) in the future, each with their own compression formats.
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#19This is really a job for JPEG-XL, which supports decode of portions of larger images and has recently been added to the DICOM standard.
No. Jpg conpression sucks. Medical data should not be compressed loosely. PNG and TIFF for the win
Re: Show HN: Streaming gigabyte medical images from S3 without downloading them
#20This is really a job for JPEG-XL, which supports decode of portions of larger images and has recently been added to the DICOM standard.
No. Jpg conpression sucks. Medical data should not be compressed loosely. PNG and TIFF for the win
unlike jpeg, jpeg-xl supports lossless compression too.