Branch coverage

[jimhester]

No description provided.

[jimhester]

This would require some thought into how to report it. Also a condition in trace_calls() to detect conditionals and switch statements.

[hyeyoungshin]

Dear Jim,

Filip Krikava and I are working on improving genthat (an automatic test extraction tool). One way to do this is to look at branch coverage and we are wondering if you are working on adding branch coverage to covr or aware of anyone doing that already. If not, we'd like to try it!

[jimhester]

No, we are not currently working on branch coverage. Thanks for offering to try and tackle it!

I am happy to help you if you have questions or run into problems trying to implement it.

[hyeyoungshin]

Great. Thank you!

[hyeyoungshin]

Dear Jim,

While tackling branch coverage, I found something that makes me wonder if you meant it to work this way or not.

Two functions that are semantically the same, but syntactically different result in a different coverage when called with the same argument. These functions contain an if statement in the body. For example, in pseudo code

f1 <- function(x) if (x > 2) x + 1 else 0
f2 <- function(x) { if (x > 2) x + 1 else 0 } ## same function but the body is wrapped in { }

code_coverage(f1, f1(0)) ## 100%
code_coverage(f2, f2(0)) ## 0 % (by taking min)

I found that the injection of covr::counter(key) is done once (in the beginning of the entire if statement) for f1 and three times for each expression (x > 2, x + 1, and 0) for f2.

print_coverage(f1_coverage) returns 100%.
print.coverage (f2_coverage, by = "expression") returns 66.67% accounting for the two expressions (if condition and else branch) hit during execution which seems more accurate to me.

I did one more experiment with two other functions: f3 and f4.

f3 <- function(x) if (x > 2) if x + 1 else { if (y > 0) x / y }
f4 <- function(x) { if (x > 2) { if ( y > 0) x / y } else 0 }

In the case of f3: covr::counter(key) is injected three times once for the entire if statement and the rest for the expressions in the { }.
In the case of f4: Can you guess? covr::counter(key) is injected four times for x > 2, y > 0, x / y, and 0.

So I concluded that it is something to do with { followed by an if.

I've been trying to find at which point the injection of covr::counter(key) is done differently for the two functions and I am still working on it.

If you have any thoughts on this and/or suggestions where to look at, please share them!

Thank you and happy holidays. :)

HY

[jimhester]

This has to do with how R creates source references. When using braces R always creates a source reference, but without it only the full expression has source references. covr only adds tracing code for expressions with a source reference, though it does some imputation of sources references as well (https://github.com/r-lib/covr/blob/master/R/parse_data.R), though as you discovered it is not perfect. See the difference between the two functions in the below

f1 <- function(x) if (x > 2) x + 1 else 0
f2 <- function(x) { if (x > 2) x + 1 else 0 }

str(f1)
#> function (x)  
#>  - attr(*, "srcref")= 'srcref' int [1:8] 1 7 1 41 7 41 1 1
#>   ..- attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800>
str(body(f1))
#>  language if (x > 2) x + 1 else 0

str(f2)
#> function (x)  
#>  - attr(*, "srcref")= 'srcref' int [1:8] 2 7 2 45 7 45 2 2
#>   ..- attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800>
str(body(f2))
#>  language {  if (x > 2); x + 1; else 0 }
#>  - attr(*, "srcref")=List of 2
#>   ..$ : 'srcref' int [1:8] 2 19 2 19 19 19 2 2
#>   .. ..- attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800> 
#>   ..$ : 'srcref' int [1:8] 2 21 2 43 21 43 2 2
#>   .. ..- attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800> 
#>  - attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800> 
#>  - attr(*, "wholeSrcref")= 'srcref' int [1:8] 1 0 2 45 0 45 1 2
#>   ..- attr(*, "srcfile")=Classes 'srcfilecopy', 'srcfile' <environment: 0x7fe94f8b2800>

covr:::trace_calls(f1)
#> function (x) 
#> if (TRUE) {
#>     covr:::count("<text>:1:7:1:41:7:41:1:1")
#>     if (x > 2) 
#>         x + 1
#>     else 0
#> }

covr:::trace_calls(f2)
#> function (x) 
#> {
#>     if (if (TRUE) {
#>         covr:::count("<text>:2:25:2:29:25:29:2:2")
#>         x > 2
#>     }) 
#>         if (TRUE) {
#>             covr:::count("<text>:2:32:2:36:32:36:2:2")
#>             x + 1
#>         }
#>     else if (TRUE) {
#>         covr:::count("<text>:2:43:2:43:43:43:2:2")
#>         0
#>     }
#> }

^{Created on 2019-12-18 by the reprex package (v0.3.0)}

In practice this comes up rarely as most non trivial functions use braces for their function bodies in R. but it still would be nice to include these cases in the imputation code and make it more robust if possible.

[hyeyoungshin]

Aha! It has something to do with srcref. I will look into parse_data.R. More specifically, how impute_srcref handles the cases. Thank you, Jim, for taking a look at this and giving me a direction!

HY

[jangorecki]

Hi Jim, thanks for great package.

In practice this comes up rarely as most non trivial functions use braces for their function bodies in R.

It affects not only functions but simple if 0 else 1 branches as well.
I use it commonly, makes code more tidy, easier to read.

nul = if (FALSE)
  0L         ## falsely covered
else
  1L

nul = if (FALSE) {
  0L         ## correctly not covered
} else {
  1L
}

If solving this issue is not on the roadmap anytime soon, maybe you could put this example to FAQ?

[hyeyoungshin]

Hi Jim, thanks for great package.

In practice this comes up rarely as most non trivial functions use braces for their function bodies in R.

It affects not only functions but simple if 0 else 1 branches as well.
I use it commonly, makes code more tidy, easier to read.

nul = if (FALSE)
  0L         ## falsely covered
else
  1L

nul = if (FALSE) {
  0L         ## correctly not covered
} else {
  1L
}

If solving this issue is not on the roadmap anytime soon, maybe you could put this example to FAQ?

Hi, we are working on this issue in the covr version with branch coverage.