Skip to content

FEAT: add a code coverage tool

Eugène Planteur requested to merge issue#657/code_coverage_analysis_tool into develop

As proposed in issue #657, add a code coverage analysis tool. After running tests with the coverage option and producing json files for the develop branch and a branch to merge, the user runs this code to compare the difference of code coverage of the tests between what is already done on develop and the new branch. Two outputs are possible: a table printed in the terminal and an HTML file containing a table. This table contains the name of .py files, the current code coverage on develop, the code coverage of the new branch and the difference between them. For example:

File Difference Develop coverage Incoming merge coverage
file1.py -5 % ↓ 80 % 75 %
file2.py 6 % ↑ 70 % 76 %
file3.py 40 % new file 40 %
file4.py 95% new file 95 %

Four cases are possible:

  • The file already exists in develop:
    • The coverage is worse than what's already done in develop: file1 (not acceptable)
    • The coverage is the same or better: file is "acceptable": file2 (acceptable)
  • The file doesn't exists yet in develop:
    • The coverage doesn't meet the requirements (a threshold can be given before the analysis, 75% by default): file3 (not acceptable)
    • The coverage meets the requirements: file4 (acceptable)

The user can also provide an "ignore threshold": some small changes in a file (adding/removing one line ...) may induce small changes in code coverage, which may represent a real downgrade of the coverage (0.2% by default).

The inputs are:

  • develop_coverage: Path to the develop's code coverage json file
  • merge_coverage: Path to the branch's code coverage json file
  • new_files_threshold: Threshold for new files' code coverage
  • ignore_threshold: Threshold for ignoring small changes in files' coverage
  • html_file_path: Path to the output HTML file (if not set, the file is not produced)
Edited by Eugène Planteur

Merge request reports