Earlier quoted context omitted.
The listing is perhaps in line with the first two "s". It seems it always iterates through all files, reads the "meta.json", then filters?
Yes, indeed. The list operation is expensive. The S3 spec says that the list output needs to be sorted. 1. All filenames are read. 2. All filenames are sorted. 3. Pagination applied. It doesn't scale obviously, but works ok-ish for a smaller data set. It is difficult to do this efficiently without introducing complexity. My applications don't use listing, so I prioritised simplicity over performance for the list oper…
Listing was IMO a problem with minio as well, but maybe it is not that important because it seems to have succeeded anyway.