0

Test case title in the release pipeline Tests tab display outdated parameter values despite the actual test outcome updating and the underlying published test files (.trx) containing the correct updated parameter values.

Pipeline overview: Task runs dotnet test to generate a test file then another “Publish Test Results” task runs to publish the test file in the release

Sample test function:

[Theory]
[MemberData(nameof(TestDataProvider.GetTestData), MemberType = typeof(TestDataProvider))]
public async Task MyParameterizedTest(Dictionary<string, string> pathParams, object? queryParams = null)
{
    // Assert
}

public class TestDataProvider
{
    public static IEnumerable<object[]> GetTestData()
    {
        yield return new object[] { new Dictionary<string, string> { ["lookupId"] = "31" }, null };
    }
}

On the initial run, title of test case was showing correctly in the tests tab as:

MyParameterizedTest(pathParams: [[“lookupId”] = “31”], queryParams: null)

  • When I change the passed parameters to another value:
    public static IEnumerable<object[]> GetTestData()
    {
        yield return new object[] { new Dictionary<string, string> { ["lookupId"] = "5" }, null };
    }

Expected test case title: MyParameterizedTest(pathParams: [[“lookupId”] = “5”], queryParams: null)

Actual test case title: MyParameterizedTest(pathParams: [[“lookupId”] = “31”], queryParams: null)

Tests tab still shows the initial test title no matter the value I change despite the published test file showing the correct test name in visual studio

  • In initial run, if a test has multiple calls it would pick a single signature and use it as case title for all calls:
    public static IEnumerable<object[]> GetTestData()
    {
        yield return new object[] { new Dictionary<string, string> { ["lookupId"] = "31" }, null };
        yield return new object[] { new Dictionary<string, string> { ["lookupId"] = "32" }, null };
    }    

Expected test case titles:

MyParameterizedTest(pathParams: [[“lookupId”] = “31”], queryParams: null) MyParameterizedTest(pathParams: [[“lookupId”] = “32”], queryParams: null)

Actual test case titles:

MyParameterizedTest(pathParams: [[“lookupId”] = “31”], queryParams: null) MyParameterizedTest(pathParams: [[“lookupId”] = “31”], queryParams: null)

Any idea how to solve this, or if this is an issue with azure devops?

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.