Comparison of gaussian processes implementation
During the validation of the implementation proposed in #368 (closed), some strange behavior appears. This issue want to summarize them.
To ensure that data are exactly the same during runs, some additionals files are wrote during iota2 runs:
- the ogc_fid of samples used for learning and validation from the iota2 training set
- the inducing points randomly taken among the learning set
With this information, we are able to retrieve the exactly same input configuration for gp learning.
An external script was wrote. It use both code from iota2 (python 3.6) and @valentine-bellet contribution (python 3.8).
As this script must work with two version of python, all functions from iota2 are extracted and copied in this script: compare_gp.py
The scikit versions used are also different and not compatible, then we must compute the feature scaler in this script too. Both were compared and are identical (until 10e-8).
So we have three version to compare:
- The iota2 implementation, using python 3.6, pytorch 1.4.0 and gpytorch 1.2.0 (from conda-forge)
- The compare_gp script with python 3.6, pytorch 1.4.0 and gpytorch 1.2.0 (from conda-forge)
- The compare_gp script with python 3.8, pytorch 1.4.0 and gpytorch 1.2.0 (no information about channels as they are provided by cluster team)
Environment | iota2 | 3.6 | 3.8 |
---|---|---|---|
loss | |||
metrics |
Currently a run using compare_gp script but the GP class for iota2 is running.