NUnit 3 Assert.That throws NullReferenceException when values are not null

Question

This is driving me crazy! I am using NUnit version 3.6.0 and I am getting unexpected results from my tests. I have created a really simple test to demonstrate the problem I have:

[TestFixture]
public class NunitTest
{
    [Test]
    public void TestIt()
    {
        string x = "x";
        string y = "y";

        // this fails (expected) but with NullReferenceException (unexpected)
        Assert.That(x, Is.EqualTo(y));
    }
}

When the test runs I get a NullReferenceException when I am expecting something like "expected is 'X', actual is 'Y'"

I have furthered the asserts and they all pass

[TestFixture]
public class NunitTest
{
    [Test]
    public void TestIt()
    {
        string x = "x";
        string y = "y";

        // this passes
        bool atest = x.Equals(y);
        Assert.IsFalse(atest);

        // this passes
        Assert.IsNotNull(x);
        // this passes
        Assert.IsNotNull(y);

        // this fails (expected) but with NullReferenceException (unexpected)
        Assert.That(x, Is.EqualTo(y));
    }
}

The other weird thing is that when the values are equal the test passed correctly. Also worth noting I get exactly the same problem when I use Assert.AreEqual(x, y)

Stack Trace:

   at NUnit.Framework.Assert.ReportFailure(String message)
   at NUnit.Framework.Assert.ReportFailure(ConstraintResult result, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression, String message, Object[] args)
   at NUnit.Framework.Assert.That[TActual](TActual actual, IResolveConstraint expression)
   at VSI.Tests.NunitTest.TestIt() in c:\Users\DavidBecker\Documents\Visual Studio 2013\Projects\VSI.Tests\VSI.Tests\NunitTest.cs:line 30

Tools:

.NET Framework version 4.5.1
Visual Studio 2013 Professional
Resharper Version 9

EDIT: I am running the tests from Resharper.

It's starting look like Resharper being the issue. If I run the tests in NUnit.ConsoleRunner.3.6.0 it behaves as expected. I have other tests in NUnit 3.2.1 that work fine with the Resharper runner so I think the combo of RS/NUnit.3.6.0 is the culprit — Dave Becker
– Dave Becker, Commented Feb 15, 2017 at 12:43

Chris · Accepted Answer · 2017-02-15 17:20:17Z

2

You're right that ReSharper 9 is the problem.

ReSharper didn't support NUnit 3 until ReSharper version 10. There are however are a number of similarities between NUnit versions 2 and 3, which mean that some older NUnit 2 runners will appear to run NUnit 3 tests - however, this won't be done accurately, and is neither supported or advised.

In the final version of NUnit 2 (2.6.4) - an exception was put in, such that a runner would error if attempting to load NUnit 3 tests would fail immediately, instead of attempting to run tests in an unsupported manner. It seems likely however that ReSharper 9 is using a runner older than NUnit 2.6.4, which doesn't throw this error.

The reason this has only become apparent to you with NUnit 3.6 is due to an internal change with how assertions are registered within NUnit. The coincidental previous success now no longer succeeds - although in many ways, this is an advantage, as it gets rid of this right-side failure, where tests appear to be passing, but aren't actually running correctly.

ReSharper supports NUnit 3 from version 10, although that was the point I believe when they switched from a one-off purchase model to a subscription model. As another option - consider the NUnit 3 Test Adapter (NuGet package or VSIX extension) - which allows you to run NUnit tests individually, using Visual Studio's own test window.

(If your interested in further info, this issue came up recently over on GitHub also: https://github.com/nunit/nunit/issues/1992)

answered Feb 15, 2017 at 17:20

Chris

6,0722 gold badges37 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Dave Becker Over a year ago

thanks for detailed explanation. To be honest I'm not bound in any way to use RS for unit testing, it's just convenient. I'll definitely take a look at NUnit 3 Test Adapter.

Chris Over a year ago

No problem. We use ReSharper 10, but find the Resharper NUnit runner in it quite buggy, and often end up relying on the NUnit console runner or adapter instead. It's a shame, because Resharper has the nicest interface, just the least success at actually running things!

Dave Becker · Accepted Answer · 2017-02-15 12:59:33Z

0

Running NUnit.3.6.0 tests in Resharper (9) Unit Test Runner has unexpected results. Use NUnit.ConsoleRunner.3.6.0 instead.

(this worked for me, if anyone has better way or can make it work in RS9 then please let me know).

answered Feb 15, 2017 at 12:59

Dave Becker

1,4331 gold badge13 silver badges24 bronze badges

1 Comment

Dave Becker Over a year ago

I did intend to mark this (mine) as the answer but Chris' explanation above goes way beyond what I know and therefore much more useful

Collectives™ on Stack Overflow

NUnit 3 Assert.That throws NullReferenceException when values are not null

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related