Think about Tests
Today, there are a lot of use cases for iota2, which are not fully handled by the already large number of tests.
Tests are split in two parts:
- unary tests for testing functions and features
- integration tests which launch the chain with a particular configuration
With all iota2 evolution, data was added according to the needs in the folder data
.
In addition, more data are stored in a remote archive as it is heavy for a git repository.
On the other part, some old tests are not really efficient, and errors were introduced with no visible impact on results.
For instance, in one configuration file, a parameter named sample_augmentationg
was found, instead of sample_augmentation
.
Some API changes have been seen in OTB but not applied in iota2... these kind of changes must be detected by tests...
Another example is ClassificationShapingTest, where the label decoding is not tested at all.
Maybe there are other tests which requires updates, and this should be verified.
The aim of this issue is to think about the best solution to adopt.
A first proposal:
-
Maybe, we can consider #236 (closed) as a starting points, and introduce a code coverage metric.
-
Then evaluate how reduce the amount of different data and organize them better than currently.
-
And finally ensure that all tests are efficient (and robust?)
Refactoring test can be a substantial task (maybe we can split the work or do it per iteration), and must be done carefully to not loose in coverage.