Validation by geographic strata
Currently, the validation is done in a global way on all the treated tiles.
An improvement could be to realize, instead or in addition, a validation by eco-climatic region.
I see three options:
- At the end of the workflow using input region vector
If the user provide a region vector file as input,the algorithm could be:
- the region label must be added to the reference data validation sets
- for computing the confusion matrix, the samples must be split by region before, and then repeat the computation for each region/tile
- Outputs could be: one text file like RESULTS.txt for each region, and the corresponding matrix images. Or another kind of output ?
If the user not provide region vector file, the current validation is done
-
At the end of the workflow using an additional file (can be a duplicate of region vector) Another solution is to add a parameter dedicated to stratified validation, allowing the user to provide any kind of vector file. In this case, a first step is to intersect the validation area with the validation polygons. For each different area, validation outputs will be produced.
-
After the training step Related to #336 , #325 , #332 (closed) Once the model is train, it can be applied directly to the validation samples. In this case it is quite simple to produce several kind of metrics with different scale: region, tile, global,..., according to the region file (1) or an additional file (2)
The major drawback of this option is the need to store also the samples for validation. The validation on samples, should be an option, and if disabled the current validation will be done.
The large part of work is already done for regression #325, the missing part is the validation procedure and the possibility to choose the validation scale.
@inglada , @mfauvel can you correct this issue, if it deviates from your initial idea?
@ArthurV , @thierionv if you have any ideas or needs on this subject...