Categories aren't shown in Visual Studio when MTP is on

[victorburckel]

Hi,

Don't know if this is a nunit adapter issue, but I tried switching to Microsoft Testing Platform, and categories don't show up in the editor anymore:

Is this an issue with the adapter ?

Here is my simple test code:

namespace NUnitMTP
{
    public class Tests
    {
        [Test, Category("Cat1")]
        public void Test1()
        {
            Assert.Pass();
        }

        [Test, Category("Cat2")]
        public void Test2()
        {
            Assert.Pass();
        }
    }
}

and my csproj is as follows:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    
    <EnableNUnitRunner>true</EnableNUnitRunner>
    <OutputType>Exe</OutputType>
    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.collector" Version="6.0.4">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
    <PackageReference Include="NUnit" Version="4.3.2" />
    <PackageReference Include="NUnit.Analyzers" Version="4.6.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="NUnit3TestAdapter" Version="5.0.0" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="NUnit.Framework" />
  </ItemGroup>

</Project>

[OsirisTerje]

I can confirm this.
I also checked what we get back from the framework, and it does include the categories:

I have uploaded the repro here: https://github.com/nunit/nunit3-vs-adapter.issues/tree/master/Issue1258

@Evangelink I didn't think the MTP did any changes to the processing of the categories, that should be entirely on the adapter side. Do you understand what is going on here ?

[OsirisTerje]

According to @Youssef1313 this is a VSTestBridge issue, and a fix is under way here: microsoft/testfx#5409

[Youssef1313]

Looking again, this might have been also fixed by microsoft/testfx#5316. @OsirisTerje Can you try latest MTP? Instructions can be found at https://aka.ms/mstest/preview (the doc feels like it's MSTest but it applies to MTP)

[victorburckel]

Not sure I did the test correctly, but I tried to force <PackageReference Include="Microsoft.Testing.Extensions.VSTestBridge" Version="1.7.0" /> in the previous dummy project to switch to the latest release, but it seems like I still get the issue.

[OsirisTerje]

@nohwnd Just checked this with the latest alpha.3 which includes the 1.7.0 bridge, and no change - still no categories showing in TE.

[Youssef1313]

@OsirisTerje It looks like the TestCase instance we receive from NUnit doesn't have Traits set.

[Youssef1313]

Okay looks like you are using your own VSTest property rather than the more standardized Traits.

nunit3-vs-adapter/src/NUnitTestAdapter/CategoryList.cs

Line 35 in 204486c

internal static readonly TestProperty NUnitTestCategoryProperty = TestProperty.Register(

In VSTestBridge, we will probably need to handle this obsolete flag.

[OsirisTerje]

@Youssef1313
We do use Traits, but for NUnit properties. The Category is a special one, which maps to the VSTest TestCategory.

It could be possible to add that as a trait too, but in order to maintain VSTest compatibility, we would have to know that we are called from MTP. I see that is being done for execution, but not for discovery.