41–50 of 51 posts
Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image.
Wish I knew how to commercialise it…
I did something similar once for a mining technique called “core logging”. It’s a single photo about 1000 pixels wide and several million “deep”: what the earth looks like for a few km down. Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image. Wish I knew how to commercialise it…
I did something similar once for a mining technique called “core logging”. It’s a single photo about 1000 pixels wide and several million “deep”: what the earth looks like for a few km down. Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image. Wish I knew how to commercialise it…
I'm curious about the "core logging" photo. Where can I find one? Do you have an implementation of your solution? I would be curious to have a look at it.
In all seriousness, how is it not useful for gold mining or phracking?
The generalized form of this range-request-based streaming approach looks something like my project VirtualiZarr [0]. Many of these scientific file formats (HDF5, netCDF, TIFF/COG, FITS, GRIB, JPEG and more) are essentially just contiguous multidimensional array(/"tensor") chunks embedded alongside metadata about what's in the chunks. Efficiently fetching these from object storage is just about efficiently fetching t…
Yeah, a recurring thought is that these should condense into Apache Arrow queried by DuckDB but there must be some reason for this not to have already happened.
This is really a job for JPEG-XL, which supports decode of portions of larger images and has recently been added to the DICOM standard.
No. Jpg conpression sucks. Medical data should not be compressed loosely. PNG and TIFF for the win
Please don’t use AWS S3 there’s vast numbers of much cheaper compatible choices.
I guess by "compatible" you mean the data plane. There are choices that speak the S3 data plane API (GetObject, ListBucket, etc). There are no alternatives that support most of the AWS S3 functionality such as replication, event notifications.
I did something similar once for a mining technique called “core logging”. It’s a single photo about 1000 pixels wide and several million “deep”: what the earth looks like for a few km down. Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image. Wish I knew how to commercialise it…
You've already done the "building v1" part, and have started to do the "talking about it" part.
Next step is to write up how one could use it, how it is better than the alternatives, and put it up on a website.
I'm happy to chat about it if you like. My email is in my profile.
Once you have real users, they will pull the v2 out of you, and that will be what you'll sell.
What I've written above sounds like a business proposition, but I want to clarify that I'm just offering to share what I know for free :-)
Interesting - I'm not so familiar with S3 but I wonder if this would work for WSI stored on-premises. Imposing lower network requirememts and a lightweight web viewer is very advantageous in this use case. I'll have to try it out!
When WSI are stored on-premise, they are typically stored on hard drives with a filesystem. If you have a filesystem, you can use OpenSlide, and use a viewer like OpenSeaDragon to visualize the slide. WSIStreamer is relevant for storage systems without a filesystem. In this case, OpenSlide cannot work (it needs to seek and open the file).
I did something similar once for a mining technique called “core logging”. It’s a single photo about 1000 pixels wide and several million “deep”: what the earth looks like for a few km down. Existing solutions are all complicated and clunky, I put something together with S3 and bastardised CoGeoTIFF, instant view of any part of the image. Wish I knew how to commercialise it…
I'm curious about the "core logging" photo. Where can I find one? Do you have an implementation of your solution? I would be curious to have a look at it.