NUnit API And Running Tests Programmatically

Question

This is the first time I'm trying to hack into the NUnit interface, and am facing some issues.

This is what I am trying to achieve:

Take a set of string inputs as code files and compile them into an assembly
take the in-memory assembly and pass it to NUnit for identifying and running the tests in this code

This is what I have so far

/// <summary> Validates whether the code files are valid </summary>
public string Validate(string[] codefiles)
{
    // To avoid NullReferenceException within NUnit
    // don't know why this is needed!
    TestExecutionContext.CurrentContext.TestPackage
        = new TestPackage("PersonTest");

    var assembly = BuildAssembly(codefiles);
    TestSuite suite = new TestAssembly(assembly, "PersonTest");
    var result = suite.Run(new NullListener(), TestFilter.Empty);
    return result.IsSuccess ? string.Empty : result.Message;
}

/// <summary> Builds an assembly </summary>
private static Assembly BuildAssembly(string[] code)
{
    var compilerparams = new CompilerParameters(new[] {"nunit.framework.dll"})
    {
        GenerateExecutable = false,
        GenerateInMemory = true
    };
    var results = new CSharpCodeProvider()
        .CompileAssemblyFromSource(compilerparams, code);

    if (!results.Errors.HasErrors) return results.CompiledAssembly;

    throw new Exception(GetErrors(results));
}

These are the two strings I am trying to compile (by sending as string array with two elements into the validate method above)

private const string File1 = @"
    public class Person 
    {
        public string Name {get;set;}

        public Person(string name)
        {
            Name = name;
        }
    }";

private const string ProperInput = @"
    using NUnit.Framework;

    [TestFixture]
    public class PersonTest
    {
        [Test]
        public void CheckConstructor()
        {
            var me = new Person(""Roopesh Shenoy"");
            Assert.AreEqual(""Roopesh Shenoy"", me.Name);
        }
    }";

Problem:

The compilation happens fine - the assembly generated even has these two types. Also not passing the nunit.framework.dll into referenced assemblies gives compilation exception (when dynamically compining), so it means that the TestFixture/Test addributes are recognized as well.

However, when I debug the TestAssembly just shows test count as zero. So instead of showing test success or failure, it shows inconclusive and basically just doesn't do anything.

I tried going through the NUnit codebase, but haven't reached any good conclusion yet - most of their convenient methods expect the assembly to be on the file system, so I'm trying to figure out exactly how to do this. Any ideas?

any reason you just can't right your assemblies to disk temporarily and pass them to nunit? — Ralph Willgoss
– Ralph Willgoss, Commented Nov 3, 2012 at 8:11
Besides the fact that I shouldn't have to do it?! I don't want to depend on existence of a file-system. — Roopesh Shenoy
– Roopesh Shenoy, Commented Nov 3, 2012 at 8:13
ok thanks, just trying to understand context before I go through the code — Ralph Willgoss
– Ralph Willgoss, Commented Nov 3, 2012 at 8:21
ok thanks for context. NUnit does some AppDomain creation when it runs, I wonder if somehow doing all this dynamically breaks that and it can't see your tests. Just a random theory since what your doing isn't the normal way, the solution might be left field — Ralph Willgoss
– Ralph Willgoss, Commented Nov 3, 2012 at 8:58
Thanks Ralph but don't bother if you haven't yet started - I figured out the problem and very nearly done; I'll post the findings here once completed. Thanks a lot for your suggestion about setting up dependencies, that was they key! — Roopesh Shenoy
– Roopesh Shenoy, Commented Nov 3, 2012 at 15:26

Ralph Willgoss · Accepted Answer · 2012-11-03 17:29:55Z

2

NUnit does some AppDomain creation when it runs, I wonder if somehow doing all this dynamically breaks that and it can't see your tests due to missing dependencies?

answered Nov 3, 2012 at 17:29

Ralph Willgoss

12.3k4 gold badges67 silver badges70 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Roopesh Shenoy Over a year ago

I'm marking this as answer to give due credit - this was very close to the main problem, which was a missing call to InstallBuiltins(). However do refer to my answer for the full code that works.

Owen Blacker · Accepted Answer · 2013-03-19 12:43:09Z

Ah it works!

I was missing a couple of steps (where the fixtures were separately identified and added to the assembly) + initially a call to "InstallBuiltIns" method. It works now and this is how it looks:

//this is needed only once
CoreExtensions.Host.InstallBuiltins();

var assembly = BuildAssembly(codefiles.ToArray());

// to avoid NullReferenceException - don't know why this is needed!
TestExecutionContext.CurrentContext.TestPackage
    = new TestPackage(assembly.GetName().FullName);

var suite = GetTestSuiteFromAssembly(assembly);
return suite.Run(new NullListener(), TestFilter.Empty);

where:

/// <summary>
/// Converts a given assembly containing tests to a runnable TestSuite
/// </summary>
protected static TestSuite GetTestSuiteFromAssembly(Assembly assembly)
{
    var treeBuilder = new NamespaceTreeBuilder(
        new TestAssembly(assembly, assembly.GetName().FullName));
    treeBuilder.Add(GetFixtures(assembly));
    return treeBuilder.RootSuite;
}

/// <summary>
/// Creates a tree of fixtures and containing TestCases from the given assembly
/// </summary>
protected static IList GetFixtures(Assembly assembly)
{
    return assembly.GetTypes()
        .Where(TestFixtureBuilder.CanBuildFrom)
        .Select(TestFixtureBuilder.BuildFrom).ToList();
}

The BuildAssembly is same as before. The only reason I had to repeat some of the logic from NUnit was that for some reason these were private within the NUnit Codebase!

Collectives™ on Stack Overflow

NUnit API And Running Tests Programmatically

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related