Nunit: [Order] does not allow duplicates on TestFixtures, documentation defect?

Created on 9 Oct 2018  ·  4Comments  ·  Source: nunit/nunit

According to the documentation: https://github.com/nunit/docs/wiki/Order-Attribute

The OrderAttribute may be placed on a test method or fixture to specify the order in which tests are run.

I am not seeing this documentation respected in two different ways.

Consider the follow TextFixture cases

[TestFixture("A", false)]
[TestFixture("B", false)]
[TestFixture("A", true)]
[TestFixture("B", true)]

This strikes me as more of a documentation/comprehension defect than anything, but the way the linked doc states "_or fixture_" implies to me that I could write

[TestFixture("A", false), Order(0)]
[TestFixture("B", false), Order(1)]
[TestFixture("A", true), Order(2)]
[TestFixture("B", true), Order(3)]

When in reality this is a compiler error for 1, 2 and 3

Duplicate 'Order' attribute

It would be really nice for this to work intuitively with duplicates, though.

All 4 comments

@Aarskin It helps both us (the NUnit devs) and users if discussion about a single issue is kept within a single gitHub issue.

The variant you are describing here is basically the same as on #3055. That is, this...

[TestFixture("A", false), Order(0)]
[TestFixture("B", false), Order(1)]
[TestFixture("A", true), Order(2)]
[TestFixture("B", true), Order(3)]

is just the same as if you had written...

[TestFixture("A", false)]
[TestFixture("A", true)]
[TestFixture("B", true)]
[TestFixture("B", false)]
[Order(3)]
[Order(2)]
[Order(1)]
[Order(0)]

as well as any other order of the eight attributes you care to use. The four TestFixture attributes are all placed on the same class, causing NUnit to instantiate that class four times. The four Order attributes are also placed on the same class, causing a conflict since a test may only have one order.

This is __not__ a limitation of NUnit. It's how the C# attribute syntax works. There are two key points to note:

  1. Any attribute in square brackets applies to the next element in the code that can accept an attribute. In this case, it's the class.

  2. A sequence of attributes in square brackets, like [A, B, C] has exactly the same meaning as the same sequence within separate pairs of brackets, like [A] [B] [C].

As a matter of style, I avoid using the comma syntax with attributes that cause multiple instances of a test to be created, because it leads to confusion.

This could be recast to allowing an Order property/parameter on the TestFixture, but before we consider that, can you give us a use case? Generally Order is used to run integration tests in a sequence where you perform parts of a longer operation in order in your tests. Parameterized TestFixture is usually used for testing the same tests with different input data. I can't think of a scenario where you would want to do that in a particular order.

My use case is logical business regression test,s and the property/parameter on the TestFixture i can't order break the way i want to use NUnit. More than before, these framework are used not only for unit test.

As a workaround, you can switch to TestFixtureSource instead of multiple TestFixture attributes. Provided there is only one TestFixtureSourceAttribute, the fixtures will be run in the order you return the data for them.

Was this page helpful?
0 / 5 - 0 ratings