FEAT: add a code coverage tool
As proposed in issue #657, add a code coverage analysis tool. After running tests with the coverage option and producing json files for the develop branch and a branch to merge, the user runs this code to compare the difference of code coverage of the tests between what is already done on develop and the new branch. Two outputs are possible: a table printed in the terminal and an HTML file containing a table. This table contains the name of .py files, the current code coverage on develop, the code coverage of the new branch and the difference between them. For example:
File | Difference | Develop coverage | Incoming merge coverage |
---|---|---|---|
file1.py | -5 % ↓ | 80 % | 75 % |
file2.py | 6 % ↑ | 70 % | 76 % |
file3.py | 40 % | new file | 40 % |
file4.py | 95% | new file | 95 % |
Four cases are possible:
- The file already exists in
develop
: -
- The coverage is worse than what's already done in
develop
: file1 (not acceptable)
- The coverage is worse than what's already done in
-
- The coverage is the same or better: file is "acceptable": file2 (acceptable)
- The file doesn't exists yet in
develop
: -
- The coverage doesn't meet the requirements (a threshold can be given before the analysis, 75% by default): file3 (not acceptable)
-
- The coverage meets the requirements: file4 (acceptable)
The user can also provide an "ignore threshold": some small changes in a file (adding/removing one line ...) may induce small changes in code coverage, which may represent a real downgrade of the coverage (0.2% by default).
The inputs are:
-
develop_coverage
: Path to the develop's code coverage json file -
merge_coverage
: Path to the branch's code coverage json file -
new_files_threshold
: Threshold for new files' code coverage -
ignore_threshold
: Threshold for ignoring small changes in files' coverage -
html_file_path
: Path to the output HTML file (if not set, the file is not produced)