Earlier quoted context omitted.
Decompress, scan as you go, discard. Having to read a few hundred GB and scan a terabyte is a nuisance. Not having to write a terabyte is priceless.
Could also maintain an in-memory index so that you can go back after the fact and extract individual files.
What might help is saving the state of the decompressor periodically, rather than just the index in the file. But that's getting pretty far into the weeds for an optimization to an infrequently used feature.