Skip to content

Analysis of product projects vs. test projects

Jump to bottom
duncanp-sonar edited this page Feb 18, 2020 · 13 revisions

Differences between analysis of test projects and product projects

The Sonar Scanner for MSBuild ("S4MSB") analyses MSBuild projects containing product code differently from projects containing test code.

Analysis of test projects

  • analysis rules are not run against test projects i.e. no issues will be reported to SonarQube/SonarCloud. This is the case even if the test project references third-party NuGet analyzer packages - those analyzers will not be executed.
  • test projects do not count towards to the Lines of Code ("LOC") limit for commercial versions of SonarQube or private SonarCloud projects.
  • metrics are not calculated for test projects, although syntax colourisation and symbol highlighting are supported.

Analysis of product projects

  • analysis rules will be run against product projects, and the issues raised will be uploaded to SonarQube/SonarCloud.
  • the SonarAnalyzer.CSharp/VB rules will be executed even if those analyzers are not referenced in the MSBuild project (S4MSB will add references to those analyzers on the fly during the build). The set of rules that is executed is determined by the Quality Profile for the Sonar project in SonarQube/SonarCloud.
  • any third-party analyzers referenced by the MSBuild project will be executed as part of the build (e.g. via NuGet package references) . Issues from those analyzers will be uploaded to SonarQube/SonarCloud as External Issues. You can configure the third-party analyzer rules as normal using a Ruleset. During the build, S4MSB will merge your custom ruleset with a ruleset generated from the Quality Profile. In the event of a conflict, the settings in the generated ruleset take precedence.

Project categorisation

S4MSB decides whether an MSBuild project contains test code or product code by looking the data in the project file. The categorisation is done at project level i.e. either the code will be treated as all product code or all test code. It is not possible to treat some of the code as test code and some as product code.

S4MSB will treat the project as containing test code if any of the following are true:

  • the project file contains the MSTest ProjectTypeGuid: 3AC096D0-A1C2-E12C-1390-A8335801FDAB
  • the project file contains the legacy Service GUID {82A7F48D-3B50-4B1E-B82E-3ADA8210C358} that is added by the Test Explorer to mark a project as a containing tests
  • the project file contains the ProjectCapability TestContainer (for new SDK-style MSBuild projects)
  • the project file name matches the RegEx set in the property sonar.msbuild.testProjectPattern, which by default is .*Tests?\.(cs|vb)proj$

There are a few special project types for which MSBuild will create and build a temporary project (e.g. Microsoft Fakes, WPF) as part of the "main" build. Such temporary projects are ignored by S4MSB. The "main" project will be categorised and treated as normal.

Explicit setting the project type

It is possible to explicitly mark a project as being a test/product project by setting the MSBuild property SonarQubeTestProject to true or false e.g.

<PropertyGroup>
  <!-- Project is not a test project -->
  <SonarQubeTestProject>false</SonarQubeTestProject>
</PropertyGroup>

Setting this property takes precedence over the default project categorisation behaviour.

Understanding why a project was categorised in a particular way

S4MSB writes information about the project categorisation to the output log. The information will appear in logs at Normal verbosity or greater. NB this logging was added in S4MSB v4.7.

Examples

The following examples are taken from the analysis of the SonarLint for Visual Studio repo:

...
SonarQubeCategoriseProject:
  Sonar: (Progress.csproj) Categorizing project as test or product code...
  Sonar: (Progress.csproj) Project categorized. SonarQubeTestProject=False
...

...
SonarQubeCategoriseProject:
  Sonar: (Progress.TestFramework.csproj) Categorizing project as test or product code...
  Sonar: (Progress.TestFramework.csproj) SonarQubeTestProject has been set explicitly to true
  Sonar: (Progress.TestFramework.csproj) Project categorized. SonarQubeTestProject=true
...

...
SonarQubeCategoriseProject:
  Sonar: (SonarQube.Client.Tests.csproj) Categorizing project as test or product code...
  Sonar: (SonarQube.Client.Tests.csproj) project is evaluated as a test project based on the project name
  Sonar: (SonarQube.Client.Tests.csproj) Project categorized. SonarQubeTestProject=True
...
Clone this wiki locally