Add compression for all written raster data set
Compression options are activated only for final raster. However, we can save a lot of space by activating them for all raster images actually written by iota2. For instance, I have observed that mask stack geotiff can be compressed from 8 Gb to about 300 mb using extended filename with OTB.
Below some code using OTB, but similar things can be done with gdal or rasterio:
# Get Mask Time Series
mask_stack, _, n_dates = s2_instance.get_time_series_masks(ram=ram)
mask_stack.SetParameterString(
"out",
mask_stack_filename
+ "?&gdal:co:COMPRESS=ZSTD&gdal:co:PREDICTOR=2&gdal:co:BIGTIFF=YES&gdal:co:NUM_THREADS=ALL_CPUS",
True,
)
mask_stack.ExecuteAndWriteOutput()
nb_dates = mask_stack.GetImageNbBands("out")
The compression options were set accordingly to the discussion about gdal compression here: https://kokoalberti.com/articles/geotiff-compression-optimization-guide/
Since we are reading image by (set of) line(s), I suspect the TILE option is not appropriated.