PartCover - Coverage of unexecuted methods

 

During the last weeks I received several emails pointing me to a potential issue in my tool ReportGenerator. The problem is that methods that have not been executed during a test run using PartCover, are marked as 'not visitable'.
I decided to take a closer look at the problem. In this post I will explain how ReportGenerator works and why it is not easy to create a workaround for the deficiency of PartCover.

The problem

Image a class containing two methods:

public class Dummy
{
  public void ExecutedMethod()
  {
    Console.WriteLine("Method1 has been executed.");
  }

  public void NotExecutedMethod()
  {
    Console.WriteLine("Method2 has been executed.");
  }
}

Now you create a UnitTest which only executes the first method. When you run the test together with PartCover you will get a report containing the following information:

<Method name="ExecutedMethod" sig="void ()" bodysize="25" flags="0" iflags="0">
  <pt visit="1" pos="0" len="1" fid="1" sl="5" sc="9" el="19" ec="10" />
</Method>
<Method name="NotExecutedMethod" sig="void ()" bodysize="25" flags="0" iflags="0" />

The report does not contain any coverage information about the uncovered method. That means you don't neither know which file contains this method nor the linenumbers of the method.

If you convert the report that PartCover has generated to a more readable version using my tool ReportGenerator, the second method is marked as 'not visitable'. That means you get a 100% coverage although the code has not been fully executed.
Melle Koning shows this behavior in detail in his blog.

How ReportGenerator works

ReportGenerator parses the report generated by PartCover or NCover. All code files that appear in the report are read line by line and the coverage information is retrieved from the report. The only relevant information of each line is its line number, no additional parsing is performed.
A line can have one of the following states:

  • Visited - Line has been executed 1-n times
  • Unvisited - Line has not been executed
  • Uncoverable - Line cannot be executed (e.g. an using statement, namespace declaration)

As mentioned above, the report does not contain enough information about unexecuted methods.
If you would like to complement the missing information, you would have to parse the relevant code files and search the method with the corresponding signature/name and determine its start and end line number.
But parsing a code file isn't as simple as you might expect. From the PartCover report you get the method's name and its signature. Image a method named 'Test' and a signature 'void (int, long)'. The method can be declared in the following ways:

  • Test(int x, long y)
  • Test(Int32 x, long y)
  • Test(int x, Int64 y)

As you can see, finding the method declaration in code isn't trivial (even in this simple example). You cannot perform this task with some string operations or regular expressions.

Possible workarounds

The current situation is quite dissatisfying, but you could use one of the following workarounds:

  • You could use NCover 1.5.8 instead of PartCover. Version 1.5.8 is still available for free.
  • You could use the old PartCover 2.3.0.18745 release instead of the current version 2.3.0.35109.

Both approaches will work correctly with unexecuted methods. And ReportGenerator will work seamlessly with all of the reports.

Conclusion

I do not know why the guys from PartCover have removed the support of uncovered methods in the latest version.
The main problem however is, that PartCover has not been updated for several months. In the meantime two forks of PartCover have been created. Let's see if they are developed more actively in the future.

Update

02.05.2010: I solved the problem by using the NRefactory library. Details are described in this post.

Feedly Feedly Tweet


Related posts


Comments


Rybolt

Rybolt

4/30/2010

Unfortunately, the 2 forks have went no where as well. I wonder how difficult it'd be to merge "no method exec" functionality back into the main branch?


Eirik F. Wahl

Eirik F. Wahl

3/18/2010

Thanks for super-quick support, Daniel. I have now tested with the latest version (0.6.2.0), and the report is generated in just 39 seconds! :-) Nice! This tool is really useful for us that want to automatically generate a HTML-report that includes the source code. The only thing that remains now, is to actually increase the coverage of our tests... ;-)


Daniel

Daniel

3/17/2010

@Eirik: A new version of ReportGenerator has been released. The performance is much better now.


Daniel

Daniel

3/11/2010

Hi Eirik, PartCover 2.3.0.18745 worked in my tests. 1 hour for a 3 MB report is very long. I think it should take about one minute. If you like, use the Contact form and provide me some more details. Perhaps it's an issue of ReportGenerator.


Eirik F. Wahl

Eirik F. Wahl

3/11/2010

PartCover 2.3.0.18745 has the same error when I try on my computer, but PartCover 2.2.0.36424 seems to work fine. The only problem now is, when I try to generate a HTML-report from my PartCover 2.2 report-XML which is 3.8 megabytes big, the report-generation will take 100% CPU for an hour before completing... but I guess that is another issue completely. :-)