iota2 issueshttps://framagit.org/inglada/iota2/-/issues2019-01-24T20:45:39Zhttps://framagit.org/inglada/iota2/-/issues/92NotImplementedError: Wrong number or type of arguments for overloaded functio...2019-01-24T20:45:39ZPhilippe TNotImplementedError: Wrong number or type of arguments for overloaded function 'Application_SetParameterString'Environment :
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
gcc-7.3.0
Running last version of iota2 built with OTB 6.4 spawn an error during step 3 :
Running step 3: compute validity mask by tile (1 t...Environment :
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
gcc-7.3.0
Running last version of iota2 built with OTB 6.4 spawn an error during step 3 :
Running step 3: compute validity mask by tile (1 tasks)
Computing pixel validity by tile
2018-12-07 00:00:00
2018-12-10 00:00:00
2018-12-20 00:00:00
2018-12-27 00:00:00
2018-12-07 00:00:00
2018-12-17 00:00:00
2018-12-27 00:00:00
Traceback (most recent call last):
File "/home/teledec/iota2-src-16.01.2019/scripts/Iota2.py", line 99, in launchTask
returned_data = function(parameter)
File "/home/teledec/iota2-src-16.01.2019/scripts/Iota2Builder.py", line 205, in <lambda>
t_container.append(tLauncher.Tasks(tasks=(lambda x: NbView.genNbView(x, "CloudThreshold_" + str(cloud_threshold) + ".shp", cloud_threshold, pathConf, workingDirectory), [os.path.join(pathTilesFeat, tile) for tile in tiles]),
File "/home/teledec/iota2-src-16.01.2019/scripts/Sensors/NbView.py", line 209, in genNbView
tilesStackDirectory = computeNbView(tile, wd, cfg, tilePixVal, TilePath)
File "/home/teledec/iota2-src-16.01.2019/scripts/Sensors/NbView.py", line 171, in computeNbView
cfg, outputRaster, tilePath)
File "/home/teledec/iota2-src-16.01.2019/scripts/Sensors/NbView.py", line 60, in nbViewOptical
"dataField", cfg, wMode=False, onlySensorsMasks=True)
File "/home/teledec/iota2-src-16.01.2019/scripts/Sampling/VectorSampler.py", line 201, in gapFillingToSample
mode=mode)
File "/home/teledec/iota2-src-16.01.2019/scripts/Common/GenerateFeatures.py", line 98, in generateFeatures
dateFile, realDates, mode)
File "/home/teledec/iota2-src-16.01.2019/scripts/Common/OtbAppBank.py", line 2175, in computeFeatures
featExtr.SetParameterValue("copyinput", copyinput)
File "/home/teledec/OTB/OTB/install/lib/otb/python/otbApplication.py", line 1844, in SetParameterValue
return self.SetParameterString(paramKey, value)
File "/home/teledec/OTB/OTB/install/lib/otb/python/otbApplication.py", line 1570, in SetParameterString
return _otbApplication.Application_SetParameterString(self, parameter, value, hasUserValueFlag)
NotImplementedError: Wrong number or type of arguments for overloaded function 'Application_SetParameterString'.
Possible C/C++ prototypes are:
Application::SetParameterString(std::string,std::string,bool)
Application::SetParameterString(std::string,std::string)
Thank you for your help
Joined are steps followed and problems encountered during build and run processes
[compilation_OTB_-_notes_et_erreurs.txt](/uploads/24c00482881422c33cfd06b85f97ba8d/compilation_OTB_-_notes_et_erreurs.txt)https://framagit.org/inglada/iota2/-/issues/91Nomenclature2019-02-01T12:26:18ZthierionNomenclatureNomenclature management should be improved. Regularisation and vectorisation steps need to access nomenclature information.
Several improvements:
1. Merge colour and nomenclature files
2. Add class alias
3. provide multi-level nomenclat...Nomenclature management should be improved. Regularisation and vectorisation steps need to access nomenclature information.
Several improvements:
1. Merge colour and nomenclature files
2. Add class alias
3. provide multi-level nomenclature capabilities for classification and validation steps
I suggest the following nomenclature structure (l1 = level 1, l2 = level 2)
*class name l1, code l1, colour l1, alias l1, class name l2, code l2, colour l2, alias l2*
For instance:
*Urbain,10,#b106b1,Urbain, Urbain dense,1,#ff00ff,UrbainDens*
*Urbain,10,#b106b1,Urbain, Urbain diffus,2,#ff55ff,UrbainDiff*https://framagit.org/inglada/iota2/-/issues/90Data management2019-01-18T10:22:47ZArthur VincentData managementIn order to produce classifications using Sentinel-2 sensor, some data are shared between IOTA²'s run : `*STACK.tif` and `*10M.tif` by dates. In order to save processing time it has been decided to write them on disk.
However, in large ...In order to produce classifications using Sentinel-2 sensor, some data are shared between IOTA²'s run : `*STACK.tif` and `*10M.tif` by dates. In order to save processing time it has been decided to write them on disk.
However, in large scale classifications, disk space could be more precious than processing time.
That's why I propose to add a new parameter `full_pipeline` to allow IOTA² generating features without any (or the less possible) writing on disk.
The workflow could follow the one below :
superimpose every available bands -> concatenate them (to produce a time series) -> ... -> features extractionArthur VincentArthur Vincenthttps://framagit.org/inglada/iota2/-/issues/87Add support for scikit-learn training and prediction2019-01-09T08:20:57ZJordi IngladaAdd support for scikit-learn training and prediction# Rationale
Currently, iota2 only supports the use of Machine Learning algorithms available through OTB applications. It may be interesting to have a wider choice of ML algorithms for classification and regression. scikit-learn is the m...# Rationale
Currently, iota2 only supports the use of Machine Learning algorithms available through OTB applications. It may be interesting to have a wider choice of ML algorithms for classification and regression. scikit-learn is the most popular and richer ML library of the Python ecosystem and supporting its API would improve iota2 a lot. Added to the richness of choice, it would also be easier for contributors to provide new ML approaches by following the scikit-learn API which has become a _de facto_ standard even for algorithms not included in scikit-learn.
In order to support the scikit-learn API, iota2 needs the addition of 2 alternate paths, one for the training step and another for the prediction (classification or regression). These paths will be taken when the corresponding choice of algorithm is made in the configuration file.
# Training
The training will be very similar to the current one. After the vector sampler step, the sample files can be fed to the scikit-learn algorithm in order to produce a model which is saved to a file so that it can be used in the prediction step. A function with the same API as the current training based on OTB's TrainVectorClassifier needs to be implemented.
# Prediction
The prediction step is trickier, since we need to avoid writing the feature image stacks to disk. Also, streaming capabilities are needed since a whole image stack will not fit into RAM when read as a numpy array. One solution is to manually stream regions at the output of the feature extraction step using the ExtractROI OTB application. The output of the ExtractROI application can be obtained as a numpy array using the OTB Applications API. This numpy array can be passed to the scikit-learn algorithm which will return another numpy array. This output numpy array will have only one band (the label) or maybe 2 if we get confidence or probability values and can be mosaicked in memory or directly streamed to disk using the gdal python API.
# Configuration file
We will need to refactor the corresponding section of the configuration file so that the choice of the algorihtm and its parameters is straightforward for the user and without the need of having a static list of available algorithms.
# Proposed steps
Since we are not experts in the use of scikit learn, it will be easier to start by implementing a simple example, like a Random Forest, as a proof of concept before going completely generic in terms of algorithms.2019-02-28https://framagit.org/inglada/iota2/-/issues/86segmentation fault during classification step2019-01-15T09:51:36Znicodebosegmentation fault during classification step## Problem
Classification step fails with segmentation fault:
```
2011-11-03 00:00[af507bee20cb:00017] *** Process received signal ***
[af507bee20cb:00017] Signal: Segmentation fault (11)
[af507bee20cb:00017] Signal code: Address not ma...## Problem
Classification step fails with segmentation fault:
```
2011-11-03 00:00[af507bee20cb:00017] *** Process received signal ***
[af507bee20cb:00017] Signal: Segmentation fault (11)
[af507bee20cb:00017] Signal code: Address not mapped (1)
[af507bee20cb:00017] Failing at address: 0x7ff787fe8e60
[af507bee20cb:00017] [ 0] /lib64/libpthread.so.0(+0xf5d0)[0x7ff7811435d0]
[af507bee20cb:00017] [ 1] /data/iota2/scripts/install/OTB/install/lib/libopencv_ml.so.3.4(+0x9d86e)[0x7ff74ce2586e]
[af507bee20cb:00017] [ 2] /data/iota2/scripts/install/OTB/install/lib/libopencv_ml.so.3.4(+0x9fb73)[0x7ff74ce27b73]
[af507bee20cb:00017] [ 3] /data/iota2/scripts/install/OTB/install/lib/libopencv_ml.so.3.4(+0x7569e)[0x7ff74cdfd69e]
[af507bee20cb:00017] [ 4] /data/iota2/scripts/install/OTB//install/lib/otb/applications/otbapp_ImageClassifier.so(_ZNK3otb33RandomForestsMachineLearningModelIfiE9DoPredictERKN3itk20VariableLengthVectorIfEEPd+0x9e)[0x7ff74ad35eee]
[af507bee20cb:00017] [ 5] /data/iota2/scripts/install/OTB//install/lib/otb/applications/otbapp_ImageClassifier.so(_ZNK3otb20MachineLearningModelIfidE14DoPredictBatchEPKN3itk10Statistics10ListSampleINS2_20VariableLengthVectorIfEEEERKjSB_PNS4_INS2_10FixedArrayIiLj1EEEEEPNS4_INSC_IdLj1EEEEE+0xd0)[0x7ff74ad12170]
[af507bee20cb:00017] [ 6] /data/iota2/scripts/install/OTB//install/lib/otb/applications/otbapp_ImageClassifier.so(_ZNK3otb20MachineLearningModelIfidE12PredictBatchEPKN3itk10Statistics10ListSampleINS2_20VariableLengthVectorIfEEEEPNS4_INS2_10FixedArrayIdLj1EEEEE+0x153)[0x7ff74ad2ecd3]
[af507bee20cb:00017] [ 7] /data/iota2/scripts/install/OTB//install/lib/otb/applications/otbapp_ImageClassifier.so(_ZN3otb25ImageClassificationFilterINS_11VectorImageIfLj2EEENS_5ImageIiLj2EEENS3_IhLj2EEEE25BatchThreadedGenerateDataERKN3itk11ImageRegionILj2EEEj+0x8c7)[0x7ff74ad2fb67]
[af507bee20cb:00017] [ 8] /data/iota2/scripts/install/OTB//install/lib/otb/applications/otbapp_ImageClassifier.so(_ZN3itk11ImageSourceIN3otb5ImageIiLj2EEEE16ThreaderCallbackEPv+0xa5)[0x7ff74ad30345]
[af507bee20cb:00017] [ 9] /data/iota2/scripts/install/OTB/install/lib/libITKCommon-4.13.so.1(_ZN3itk13MultiThreader17SingleMethodProxyEPv+0x7)[0x7ff769b6bc07]
[af507bee20cb:00017] [10] /lib64/libpthread.so.0(+0x7dd5)[0x7ff78113bdd5]
[af507bee20cb:00017] [11] /lib64/libc.so.6(clone+0x6d)[0x7ff78075bead]
[af507bee20cb:00017] *** End of error message ***
/usr/local/bin/entrypoint.sh: line 9: 17 Segmentation fault (core dumped) su-exec iota python "${IOTABIN}" "$@"
```
## Configuration file
```
chain:
{
outputPath:'/mnt/workspace/IOTA2_Outputs/Results_bb_balok'
remove_outputPath:True
pyAppPath:'/data/iota2/scripts'
nomenclaturePath:'/mnt/workspace/Nomenclature_balok.txt'
listTile:'D0003H0007 D0003H0008 D0003H0009 D0004H0008 D0004H0009 D0005H0007 D0005H0008 D0005H0009 D0006H0006 D0006H0007 D0006H0008 D0006H0009 D0007H0006 D0007H0007 D0007H0008 D0007H0009 D0008H0006 D0008H0007 D0008H0008 D0008H0009'
L5Path:'/mnt/workspace/sensor_data'
groundTruth:'/mnt/workspace/vector_data/groundTruth2011AESN.shp'
dataField:'class'
spatialResolution:30
colorTable:'/mnt/workspace/ColorFile_balok.txt'
ratio:0.7
}
argTrain:
{
classifier :'rf'
options :' -classifier.rf.min 5 -classifier.rf.max 25 '
}
argClassification:
{
classifMode : 'separate'
}
GlobChain:
{
proj : "EPSG:2154"
}
```
## Temporary fix
Relaunch iota from the classification step (and subset the listTile to only include tiles that have not been predicted) seems to suppress the segmentation fault.
```
chain:
{
outputPath:'/mnt/workspace/IOTA2_Outputs/Results_bb_balok'
remove_outputPath:False
pyAppPath:'/data/iota2/scripts'
nomenclaturePath:'/mnt/workspace/Nomenclature_balok.txt'
listTile:'D0003H0007 D0003H0008 D0003H0009 D0004H0008 D0004H0009 D0005H0007 D0005H0008 D0005H0009 D0006H0006 D0006H0007 D0006H0008 D0006H0009 D0007H0006 D0007H0007 D0007H0008 D0007H0009 D0008H0006 D0008H0007 D0008H0008 D0008H0009'
L5Path:'/mnt/workspace/sensor_data'
groundTruth:'/mnt/workspace/vector_data/groundTruth2011AESN.shp'
dataField:'class'
spatialResolution:30
colorTable:'/mnt/workspace/ColorFile_balok.txt'
ratio:0.7
firstStep:'classification'
}
argTrain:
{
classifier :'rf'
options :' -classifier.rf.min 5 -classifier.rf.max 25 '
}
argClassification:
{
classifMode : 'separate'
}
GlobChain:
{
proj : "EPSG:2154"
}
```
## Configuration info
* ram: 32Go
* proc: AMD® Ryzen threadripper 1900x 8-core processor × 16
* To launch iota I'm using <https://framagit.org/nicodebo/iota2-docker> with otb 6.6.1.https://framagit.org/inglada/iota2/-/issues/83Class label could be string2018-11-29T12:08:52ZMathieu FauvelClass label could be stringActually, iota2 support only numeric label. String label could be supported using similarly to sklearn: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html#sklearn.preprocessing.LabelEncoder
In that...Actually, iota2 support only numeric label. String label could be supported using similarly to sklearn: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html#sklearn.preprocessing.LabelEncoder
In that case, the app should output a file with the connection between numeric and string values (or a qgis style file ?) for the final classification maps.https://framagit.org/inglada/iota2/-/issues/82Mandatory parameters are still mandatory even if the corresponding steps are ...2018-11-29T12:03:51ZMathieu FauvelMandatory parameters are still mandatory even if the corresponding steps are not doneFor instance, if I am just interested in doing the sampling step, i.e. in the config file:
firstStep:init
lastStep:sampling
I still needs to provide parameters for argTrain and so on. I could be nice to check if the requires st...For instance, if I am just interested in doing the sampling step, i.e. in the config file:
firstStep:init
lastStep:sampling
I still needs to provide parameters for argTrain and so on. I could be nice to check if the requires steps have been selected and only then make their parameters mandatory, in order to make the config file short.https://framagit.org/inglada/iota2/-/issues/80zip in the S2 folder make iota2 fail2018-11-29T11:57:28ZMathieu Fauvelzip in the S2 folder make iota2 failIf I let the zip files in the folder, i.e., the folder contains both the compressed and uncompressed files, it makes the application crash. Problem was solved by deleting the zip files.If I let the zip files in the folder, i.e., the folder contains both the compressed and uncompressed files, it makes the application crash. Problem was solved by deleting the zip files.https://framagit.org/inglada/iota2/-/issues/79Regression in iota^22018-11-29T11:54:42ZMathieu FauvelRegression in iota^2Iota^2 is actually used to perform classification. However, all the ingredients needed to perform regression/parameter estimation are available within the otbapp.
The different steps are similar to classification (sampling, training, p...Iota^2 is actually used to perform classification. However, all the ingredients needed to perform regression/parameter estimation are available within the otbapp.
The different steps are similar to classification (sampling, training, predict and validation). However, the predicted values are continuous hence the accuracy measure should be adapted. Also, contrary to classification, it is usual to normalise (center the data and divide by the std: zero mean and unit variance).
The steps to conduct regression could be:
1. _Learning_:
- using the otbapp TrainRegression, a csv file should be provided. This file could be converted from an sqlite generate using the sampling tools used for classification. Otherwise, it should be possible to provide directly the csv file without performing the sampling from an image (for instance in the situation where the data are generated from physical model).
- Normalization of the data: the value to be predict y should be normalized per default before the training: yn = (y-\bar{y})/std(y). Also, y could be multi-dimensional.
- Usually cross-validation is performed to select the best hyperparameters of the regression methods. At least, we should be able to get from the learning step a cross-validated estimation of the regression error from the app. (I don't know what is included for cross validation, but N. Karasiak has build some python tools that link otb and scikit-learn cross validation generator).
2. _Predict_: can be done with otb app. If the normalization was performed in the training set, the method should convert the predicted value back to the original dynamic (yp*std(y)+\bar[y]).
3. _Validation_: Usually, the normalised/relative root mean square error is use to provide a qualitative value of the error/precision of the regression. If the normalization is applied, the root mean square error is already normalized. Otherwise, it should be.https://framagit.org/inglada/iota2/-/issues/77PointMatchCoregistrationModel remote module installation2018-10-29T13:06:57ZSebastienPeilletPointMatchCoregistrationModel remote module installationHi,
Following #70
I modified [generation.sh](https://framagit.org/SPeillet/iota2/blob/feature/coregistration/scripts/install/generation.sh#L147) to build PointMatchCoregistrationModel remote module during the iota2 installation.
I fo...Hi,
Following #70
I modified [generation.sh](https://framagit.org/SPeillet/iota2/blob/feature/coregistration/scripts/install/generation.sh#L147) to build PointMatchCoregistrationModel remote module during the iota2 installation.
I followed the examples already provided in the script, but if you have a comment, please go ahead !https://framagit.org/inglada/iota2/-/issues/76Use Theia L3 time series as input2019-01-09T08:17:22ZJordi IngladaUse Theia L3 time series as inputAdapt the input data ingestion and bypass gapfilling and resampling. Example products:
https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/2ae2f12f-0629-517e-958a-81df2dd4fe73Adapt the input data ingestion and bypass gapfilling and resampling. Example products:
https://theia.cnes.fr/atdistrib/rocket/#/collections/SENTINEL2/2ae2f12f-0629-517e-958a-81df2dd4fe73https://framagit.org/inglada/iota2/-/issues/75Make a conda package for iota22019-02-05T13:27:56ZJordi IngladaMake a conda package for iota2Conda offers most of python the packages needed by iota2. It also provides c++ build tools (compiler+cmake, etc.) to build OTB. It should therefore be possible to provide a iota2 install through conda. This would allow an easier distribu...Conda offers most of python the packages needed by iota2. It also provides c++ build tools (compiler+cmake, etc.) to build OTB. It should therefore be possible to provide a iota2 install through conda. This would allow an easier distribution of iota2 and would not only be limited to Ubuntu and CentOS as the current install scripts are.https://framagit.org/inglada/iota2/-/issues/74New IOTA² step definition2018-11-29T15:16:47ZArthur VincentNew IOTA² step definition#### Description
One of IOTA² main components are steps. However, steps are not well define. In fact, if a developer has to add new steps it is not easy.
That's why I propose to create the **Base class** : ``Step``
I already wrote a fu...#### Description
One of IOTA² main components are steps. However, steps are not well define. In fact, if a developer has to add new steps it is not easy.
That's why I propose to create the **Base class** : ``Step``
I already wrote a functional [code](https://framagit.org/ArthurV/iota2/tree/feature-Step_refac) which run an example of what is possible by creating sub-classes of Steps.
A new module is available : [Steps](https://framagit.org/ArthurV/iota2/tree/feature-Step_refac/scripts/Steps) which contains Step base class providing new functionalities :
1. **Automatically** find resources for HPC user
Using step's name and the configuration file dedicated to resources. If no corresponding resources are found, apply default resource (minimum ask-able)
2. **3 methods** as to be redefine to launch a complete step:
**step_execute** : This is the ``only`` method (lambda function) call to run a step
**step_inputs** : define parameters to iterate
**step_outputs** : define how to get step expected outputs
3. A new function is dedicated to collect and remove all temporary files
**step_clean** is call if a step succeed. This is the place to define the clean-up.
4. Provide a way to **connect** steps together
**step_connect** method allow to tell to IOTA² "the input of step B is the output of step B" as in the [example](https://framagit.org/ArthurV/iota2/blob/feature-Step_refac/scripts/Iota2Builder.py#L117)
Also, I created a new container class : [StepContainer](https://framagit.org/ArthurV/iota2/blob/feature-Step_refac/scripts/Steps/IOTA2Step.py#L20), which contains all **unique** steps
New methods could be added to ``Steps`` class to improve services and could be discuss in this post. Maybe, it is the time to create a new class : ``Tile`` as suggested [here](https://framagit.org/inglada/iota2/issues/5) by J.Michel. Then add *Tiles* as attributes of iota2 class ?https://framagit.org/inglada/iota2/-/issues/73VHR management for coregistration2018-11-20T15:28:30ZSebastienPeilletVHR management for coregistrationFollowing (https://framagit.org/inglada/iota2/issues/70)
In order to have to same working method, it's seems to be necessary to add some arguments in the configuration file to manage VHR data (eg "VHR_path", "VHR_ref_band" etc).
Most o...Following (https://framagit.org/inglada/iota2/issues/70)
In order to have to same working method, it's seems to be necessary to add some arguments in the configuration file to manage VHR data (eg "VHR_path", "VHR_ref_band" etc).
Most of arguments used by coregister.py will have default values. Like this, this step will not overload the configuration file.https://framagit.org/inglada/iota2/-/issues/72Inserting the coregistration step2018-10-29T12:06:54ZSebastienPeilletInserting the coregistration stepFollowing [https://framagit.org/inglada/iota2/issues/70]
From what I saw of the iota² worflow (correct me if I have wrong), the coregistration step should be inserted between the 2nd (generate common masks) and the 3rd step (compute val...Following [https://framagit.org/inglada/iota2/issues/70]
From what I saw of the iota² worflow (correct me if I have wrong), the coregistration step should be inserted between the 2nd (generate common masks) and the 3rd step (compute validity mask by tile).
Like this all the sensor data are already reprojected in the same epsg, and the bands are stacked per day.
I don't know if it's has to be a particular step or if it can be add to the 2nd step (and only triggered if the config file contains a VHR argument), what do you think about ?https://framagit.org/inglada/iota2/-/issues/70Co-registration of a time series on a reference VHSR image2019-02-08T15:38:59ZSebastienPeilletCo-registration of a time series on a reference VHSR imageIn the perspective of integrating the MORINGA workflow (code source of the prototype available here: https://gitlab.irstea.fr/raffaele.gaetano/moringa.git) in the iota2 platform, me and Raffaele have started to think of some features rel...In the perspective of integrating the MORINGA workflow (code source of the prototype available here: https://gitlab.irstea.fr/raffaele.gaetano/moringa.git) in the iota2 platform, me and Raffaele have started to think of some features related to coupling HR time series with VHR imagery. The first contribution we thought about is dedicated module to improve the co-registration between different optical image sources. This need comes from the fact that, in our experience, it is common to observe of a georeference gap between the VHR image and the HR time series, and even between two HR acquisitions (which can be significant for S2 imagery).
For this purpose, we are currently developing a feature for iota2 that should be composed of:
1. a generic OTB application to find sift points between an input image and a reference image and create a new RPC sensor model for the input image.
2. a python script to use the application and to perform the orthorectification on one or several images
The OTB application should be a adaptation of two OTB application (HomologousPointExtraction, GenerateRPCSensorModel). It inherits of some parameters of HomologousPointExtraction, fixing some others to default values for computational efficiency, such as the geobins mode. Since tie points must be output in pixel coordinates for the coregistering image, we have to slightly modify the HomologousPointExtraction application, preventing us from relying on a completely composite application solution. Several additional improvements are to be made: (1) including an option for iterative tie point search (reduces the geobins step if the number of tie points is insufficient for RPC problem solving), (2) including a resampling option if the input and the reference images have significantly different spatial resolutions (geobins at highest resolution will always be resampled to the lowest one, no matter the direction).
The python script could be made on the GenerateFeatures.py model. Like this it will be possible to launch it in different ways :
- co-registration with the OTB application between an input image and a reference image and directly apply the model to orthorectify the input image;
- co-registration with the OTB application between one image of the time series and the reference to apply the same model over the whole time series;
- separate co-registration between the reference and each image of the time series
This latter case is intended to optimize the processing of whole time series when a independent gap is observed among acquisitions at different dates. So far, in Moringa we co-locate each image of the time series separately to the VHR reference, but this approach can give problems especially when strong seasonal changes take place, which is typical on tropical agro-systems (lots of tie-points for HR images closer in time to the VHR scene, fewer and fewer when the time delay increases). To this end, we are currently investigating a cascade coregistration approach, in which only the HR image closest in date is coregistered to the VHR image, and other dates are processed in cascade using each time the already co-registered, closest HR image as the new reference.
We have already started some local developments and will soon propose some code, but any prior exchange with iota2 developers' will be welcome!Arthur VincentArthur Vincenthttps://framagit.org/inglada/iota2/-/issues/68Git - Hooks2019-01-09T08:17:56ZArthur VincentGit - HooksIn order to better-sharing IOTA²'s code between developers, I propose to introduce [hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) concept provided by GIT.
* pre-push : **every** push event would trigger one / all of t...In order to better-sharing IOTA²'s code between developers, I propose to introduce [hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks) concept provided by GIT.
* pre-push : **every** push event would trigger one / all of the following
* run all UnitTests
* run the tests on impacted files
* run code analytic
* pre-commit : on **every** commit event
* run code analytic
maybe pre-commit is too restrictive...https://framagit.org/inglada/iota2/-/issues/62iota2 installation on docker Ubuntu2018-08-31T07:24:01Znicodeboiota2 installation on docker UbuntuHello,
Thank you for iota2,
I'm trying to build a iota2 docker image. Unfortunately, Otb build fail at the opencv step:
```
Scanning dependencies of target OPENCV
[ 48%] Creating directories for 'OPENCV'
[ 49%] Performing download step...Hello,
Thank you for iota2,
I'm trying to build a iota2 docker image. Unfortunately, Otb build fail at the opencv step:
```
Scanning dependencies of target OPENCV
[ 48%] Creating directories for 'OPENCV'
[ 49%] Performing download step (download, verify and extract) for 'OPENCV'
-- OPENCV download command succeeded. See also /data/iota2/scripts/install/OTB/build/OPENCV/src/OPENCV-stamp/OPENCV-download-*.log
[ 49%] No patch step for 'OPENCV'
[ 49%] No update step for 'OPENCV'
[ 50%] Performing configure step for 'OPENCV'
CMake Error at /data/iota2/scripts/install/OTB/build/OPENCV/src/OPENCV-stamp/OPENCV-configure-Release.cmake:16 (message):
Command failed: 1
'env' 'CC=/usr/bin/cc' 'CXX=/usr/bin/g++' '/usr/bin/cmake' '-GUnix Makefiles' '-C/data/iota2/scripts/install/OTB/build/OPENCV/tmp/OPENCV-cache-Release.cmake' '-GUnix Makefiles' '/data/iota2/scripts/install/OTB/build/OPENCV/src/OPENCV'
See also
/data/iota2/scripts/install/OTB/build/OPENCV/src/OPENCV-stamp/OPENCV-configure-*.log
CMakeFiles/OPENCV.dir/build.make:109: recipe for target 'OPENCV/src/OPENCV-stamp/OPENCV-configure' failed
make[2]: *** [OPENCV/src/OPENCV-stamp/OPENCV-configure] Error 1
CMakeFiles/Makefile2:1274: recipe for target 'CMakeFiles/OPENCV.dir/all' failed
make[1]: *** [CMakeFiles/OPENCV.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
```
The Dockerfile that I'am using is the following:
```
FROM ubuntu:18.04
COPY acceptOTB.exp /data/
RUN \
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y git wget expect && \
cd /data && \
git clone -b develop https://framagit.org/inglada/iota2.git && \
apt-get install -y cmake git g++ python-dev zlib1g-dev freeglut3-dev libx11-dev libxext-dev libxi-dev libboost-all-dev swig gsl-bin libgsl0-dev python-pip python-numpy python-scipy python-matplotlib python-pandas python-pyspatialite libspatialite-dev libspatialite7 libxrandr-dev libxinerama-dev libxcursor-dev && \
pip install argparse config datetime osr dill mpi4py==2.0.0 && \
# build otb
/usr/bin/expect -f acceptOTB.exp
```
with acceptOTB.exp being:
```
#!/usr/bin/expect
set timeout -1
spawn ./iota2/scripts/install/generation.sh --all
expect "*Are you*" { send "y\r" }
expect eof
```
To build the image, copy the `Dockerfile` and `acceptOTB.exp` file in the same directory and run `$ docker build --tag=iotaubuntubuild .` (from the directory containing both files)
Notes:
* I'm not using the init_Ubuntu.sh script because [this check](https://framagit.org/inglada/iota2/blob/develop/scripts/install/init_Ubuntu.sh#L26) fails if the host is not an Ubuntu distribution.
* The expect script is used to automatically answer the question asked by the generation.sh script.https://framagit.org/inglada/iota2/-/issues/61Integrate vectorisation processing chain to IOTA².2018-08-29T08:37:13ZArthur VincentIntegrate vectorisation processing chain to IOTA².### Integrate vectorisation processing chain to IOTA² chain.
The processing chain dedicated to vectorise huge rasters is next to IOTA² scripts. As vectorisation scripts are fully operational, we can integrate them to the main chain. Use...### Integrate vectorisation processing chain to IOTA² chain.
The processing chain dedicated to vectorise huge rasters is next to IOTA² scripts. As vectorisation scripts are fully operational, we can integrate them to the main chain. Users will be able to launch vectorisation thanks to the configuration file.
the steps summary should be as the following :
```bash
Group init:
[ ] Step 1: create directories
[ ] Step 2: generate common masks
[ ] Step 3: compute validity mask by tile
Group sampling:
[ ] Step 4: generate envelopes
[ ] Step 5: Prepare samples
[ ] Step 6: merge samples by models
[ ] Step 7: generate samples statistics
[ ] Step 8: select samples
[ ] Step 9: generate samples
[ ] Step 10: merge samples
[ ] Step 11: copy samples between models according to user request
Group learning:
[ ] Step 12: learning
Group classification:
[ ] Step 13: generate classification commands
[ ] Step 14: generate classifications
Group mosaic:
[ ] Step 15: classfication shaping
Group validation:
[ ] Step 16: confusion matrix command generation
[ ] Step 17: generate confusion matrix
[ ] Step 18: confusion matrix fusion
[ ] Step 19: report generation
Group vectorisation:
[ ] Step 20: foo
[ ] Step 21: bar
...
[ ] Step N: *
```thierionthierionhttps://framagit.org/inglada/iota2/-/issues/60Pixel values of classification2018-09-06T06:42:52ZthierionPixel values of classificationWith the 30 classes OSO version, we use two values to define a class :
* code (11111, 12111, 13111, ..., 51111)
* code number (1, 2, 3, 3, ..., 30)
iota² produces the final raster *classif_seed_o.tif* with code number values. It coul...With the 30 classes OSO version, we use two values to define a class :
* code (11111, 12111, 13111, ..., 51111)
* code number (1, 2, 3, 3, ..., 30)
iota² produces the final raster *classif_seed_o.tif* with code number values. It could be interesting to get also a 16 bits raster with code values.
Vincent