Autofixture: NSubstitute.ReceivedCalls() returning wrong value using AutoFixture.AutoNSubstitute

Thanks for firing the issue. Actually, this issue has nothing to do with AutoFixture, while we are affected by it as well 😕

This issue is mostly caused by xUnit and if you rewrite the test as following, it will pass:
```c#
[Theory, AutoNSubstituteData]
public void DoItNow_UsingAutoNSubstitute_CreateManually(IFixture fixture)
{
// Arrange
var runSpeed = fixture.Freeze();
var sut = fixture.Create();
runSpeed.Speed.Returns(49);

//Act
sut.DoItNow();

//Assert
var tmp = runSpeed.Received(1).Speed;
Assert.Single(runSpeed.ReceivedCalls());

}
```

When you run tests, xUnit tries to format the test name for you. If it finds that type is not known and doesn't have ToString() overload, it uses the structural inspection and recursively fetches the property values. If you check the test name, you'll see the following:

As you might notice, xUnit fetched the Speed property value to nicely show you the test argument. As this is a usual invocation, NSubstitute counted that as a call.

It's quite strange that you didn't have this issue with v3, as nothing changed in that regards. I've just tested xUnit2 + AutoFixture v3 and still got the issue. Probably, you have upgraded the xUnit as well.

As for a workaround, I have good and bad news. The good news is that this will be resolved in the next major version of NSubsitute, as it started to override the ToString() method to return a proxy id. As result, xUnit doesn't touch the properties anymore:

The bad news is that NSubsitute v4 has not been released yet.

I could suggest you either:

• postpone migration till NSubsitute v4 is released;
• use the latest master source code of NSubsitute, make a local build and use your own NSubsitute.dll instead of NuGet package. After v4 is released, you can switch to the NuGet package.
• if you updated xUnit version as well (as this might explain why you didn't have this issue before), revert that change and use the previous version of xUnit.

I apologize for the inconvenience, but, unfortunately, we can do nothing on AutoFixture side to fix this issue.

just tested this. finally our tests asserting ReceivedCalls went back to have a successful result again.
Something that still fails are the asserts on Received(x) like Received(1).Speed Daniel mentioned above.

Do you also have an answer / solution for hat?

Are you guys working on the same project? :) If so, could you clarify how you fixed the issue?

The options suggested above should help with both the issues. If they don't - clarify the scenario.

not actually the same project but at the same company.
we've been on xunit 1.9 + nsubstitute 2 / 3 for a long time and finally managed it to get our internal assemblies (nuget packages so a simple way back to an older version of just one reference is not as much as easy as it sounds to be) to be compatible to xunit2. and now we have these two problems.

We already tried out some things and finally thought it must have something to do with autofixture - because of the frozen-attribute and a hard coded test execution count - or something like that.
after daniel opened this case he postet me the link to it.

just as you described above I replaced the nuget 3.1 reference with the one from the fresh compiled nsubstitute project. after that I now have 118 instead of 178 failing tests because of Receiced(x) is still evaluating the wrong number of calls.

my project is currently based on .net 4.5.2 with xunit2, current repo version of nsubstitute and autofixture 4.2 referenced. all of the tests are failing for the same two reasons - also there are still some receivedcalls failing in the unit tests.
I think I have to have a deeper look into it again when I'm back to the office again (out for easter).
maybe @dklinger meanwhile can describe the scenario.

@dklinger @evilbaschdi Please follow up after you have a chance to check that - it's quite interesting why you sill see the issues, even after you applied the patch.

Re-open the issue to indicate that we still have ongoing investigation.

Ok, so I've tested the last few hours. First: Thanks for your reply, your explanation and the suggestions - that helped a lot.
I've tested v4 of NSubstitute. It works for my example code above, but does not resolve the issue entirely in our real world projects. The issue is, that it resolves the problem of the wrong ReceivedCalls-count only for SUT-Calls to methods, not to properties.

Example of our now working code:

        [Theory, AutoNSubstituteData]
        public void DoItNow_UsingAutoNSubstitute(
            [Frozen] IRunSpeed runSpeed,
            GetToDaChoppa sut)
        {
            // Arrange
            runSpeed.Speed.Returns(49);

            //Act
            sut.DoItNow();

            //Assert
            var tmp = runSpeed.Received(1).Speed;
            Assert.Equal(1, runSpeed.ReceivedCalls().Count());
        }

But if I change the method "GetToDaChoppa.DoItNow();" to be a property "GetToDaChoppa.DoItNow" the ReceivedCallsCount is +1 again:

        [Theory, AutoNSubstituteData]
        public void DoItNow_UsingAutoNSubstitute(
            [Frozen] IRunSpeed runSpeed,
            GetToDaChoppa sut)
        {
            // Arrange
            runSpeed.Speed.Returns(49);

            //Act
            var x = sut.DoItNow;

            //Assert
            var tmp = runSpeed.Received(1).Speed;
            Assert.Equal(1, runSpeed.ReceivedCalls().Count());
        }

I think it again has to do with the xUnit naming as we get this one for the working method-implementation:

And that one for the non-working property-implementation:

It looks like xUnit is calling the SUT-property to retrieve its value to use it for the test-name but it is not doing this for SUT-methods. My first thought was that it has to do with the return value of our method "GetToDaChoppa()" which was void. But even after changing it to be "public int GetToDaChoppa()" xUnit is still not calling it to get a return value for the test Name. It is only a problem using properties.

For now I am stuck again. I totally agree that this is not an issue of AutoFixture. But in your opinion, knowing all of the packets better than we do, what would be your suggestion?

• opening an issue for NSubstitute?
• opening an issue for xUnit2?
• or something totally new?

@dklinger Thanks for the follow-up! Could you please share MCVE of code for the last scenario that still fails? Just to ensure that nothing is missed and I haven't misunderstood the conditions.

After that I'll try to investigate why that happens and how to work around that.

Thanks.

Yep, sure. Here it is:

System under Test:

    public interface IRunSpeed
    {
        int Speed { get; }
        void Dude();
        int Dude2();
    }

    public class GetToDaChoppa
    {
        private readonly IRunSpeed _runSpeed;

        public GetToDaChoppa(IRunSpeed runSpeed)
        {
            _runSpeed = runSpeed ?? throw new ArgumentNullException(nameof(runSpeed));
        }

        public int DoItNow
        {
            get
            {
                var runningSpeed = _runSpeed.Speed;
                return 0;
            }
        }
    }

Test-Case:

        [Theory, AutoNSubstituteData]
        public void DoItNowAsProperty_UsingAutoNSubstitute(
            [Frozen] IRunSpeed runSpeed,
            GetToDaChoppa sut)
        {
            // Arrange
            runSpeed.Speed.Returns(49);

            //Act
            var x = sut.DoItNow;

            //Assert
            var tmp = runSpeed.Received(1).Speed;
            Assert.Equal(1, runSpeed.ReceivedCalls().Count());
        }

Currently I'm trying to catch up on the discussions at https://github.com/xunit/xunit/issues/1386 and https://github.com/AutoFixture/AutoFixture/issues/805. Maybe we're doing something wrong or there is a way to tell xUnit to not autogenerate the test name.
I also tried to create my own TheoryAttribute overriding the DisplayName property but that does not help either as xUnit is in some way still using the autogenerated name internally. But that's just fyi and totally unrelated to the demo-code above.

        public sealed class MyTheoryAttribute : TheoryAttribute
        {
            public MyTheoryAttribute([CallerMemberName] string memberName = null)
            {
                DisplayName = "MyTestCase";
            }
        }

Hello, me again :)
It is indeed xUnit2 calling all Properties and Fields of test-method-parameters which are of a Complex Type. The line of code is this one: https://github.com/xunit/assert.xunit/blob/2b70a9b0c5bb291f98472ec24cec437acf8d65c8/Sdk/ArgumentFormatter.cs#L156

Thanks for your support. I've reopend an issue at the xUnit-Repo where the discussion should go on: https://github.com/xunit/xunit/issues/1682

@dklinger Thanks for the detailed scenario. It's indeed quite tricky and it's hard to somehow work around it. From AutoFixture and NSubsitute perspective there no difference whether code is called somewhere deeply inside xUnit or in the test body.

Usually, as a workaround you can use clear feature of NSubstitute:

runSpeed.ClearReceivedCalls();

This code should be run at the prologue of each test where you verify the exact number of calls. It works fine if you have a few tests, however obviously, if thousands of tests are affected, it will not help much 😅

It's a pity that NSubsitute + xUnit2 + AutoFixture integration doesn't work well and suffers from this kind of issues. AutoFixture product was designed to simplify the life, rather than to make it a nightmare 😕 Hope guys from xUnit will advice you a quick way to solve the issue for entire project.

Let me know if you believe we can do something from our side to improve the situation.