Enable coverage reporting during PR testing.
This makes the Jenkins fail... But beside that.. +1
Still working on this...
Realized that @ralph was right in lowering the fail_under value to something that can be reached.
I've excluded a few boilerplate files from coverage measurements. As soon as pagure takes the push, Jenkins should pass with a 53% coverage.
1 new commit added
Exclude boilerplate code from coverage measurement
Try cobertura reporting
3 new commits added
Jenkins: run unit tests and archive coverage report
This file shouldn't be ignored, it could contain some important code in future.
Currently it only has __version__, which is more like metadata.
__version__
The idea was to omit all the files for which currently there is no point expecting some unit tests to be implemented. That is, I wouldn't file an issue saying 'write tests for init.py' right now.
In case these files are changed in the future, so that unit testing makes sense for them, they could be removed from the omit list.
rebased onto db6ef6a66235917c641eab6b1f28b8d5d1a801ac
I agree, we can add it again in the future.
Any reason to have coverage printed in log and HTML generated?
@lholecek the HTML report is archived and offers the possibility to see results for individual files.
I was trying to use coverage.xml with the Cobertura plugin from Jenkins, to have nicer reporting, but no luck with that yet.
coverage.xml
I'll give some more tries, though. If I manage to make it work, and offers the same details as the HTML format, I'll remove the later.
The coverage for this file is currently 100%. Ignoring it now won't increase coverage.
Try cobertura again
My bad, you are absolutely right @lholecek.
I've amended the commit to remove it from the omit list. Thanks for noticing it!
Cool, look like this works.
Yep. Let's keep the HTML report, too, if you don't mind. The Cobertura report does not show a line-per-line breakdown (as coverage.xml does not include the lines).
This can be merged to stage 'Perform functional tests', the coverage would be much better.
rebased onto 62c9ddccadb2fab29c6a22830c08f3fa036ef7f7
Let's keep the HTML report
Yes, definitely (I didn't notice the HTML being used before).
Hm... this would work until the functional-tests are run like today; that is, greenwave is not deployed as an app in the test env but it's running on the slave.
An another reason, I would not do this, is that the functional tests should aim to cover functionality and treat the application as a black box, while unit tests should be the ones aiming for code coverage. Hope this makes sense :)
the functional tests should aim to cover functionality and treat the application as a black box, while unit tests should be the ones aiming for code coverage
OK, makes sense.
+1
Pull-Request has been merged by lholecek
Enable coverage reporting during PR testing.