Autofixture: Generate an object with complex constraints

Sorry for the radio silence. We are alive and I'll reply soon - am extremely busy on my primary work these days. Also working on NSubstitute v4 release, so time is very limited resource :pensive: The question is hard, so want to think about all the possible ways before posting the answer.

Thanks for the patience, keep tunes :wink:

Hi,
Any news on that ?
No pressure (I know the drill 😄 , plus it's not really blocking, I'd just really like some educated advice), it's just to know if you have some visibility.
Thanks a lot!

Good day! Finally I allocated a bit of type to answer - sorry for the hugely belated reply 😊

First of all pay attention that AutoFixture's core is pretty simple and we don't have built-in support for the complex trees with constrains. In short, the creation strategy is like following:

• Look for a public constructor or a static factory method (static method returning an instance of the current type).
• Resolve the constructor arguments and activate instance.
• Fill the writable public properties and fields with generated values.

With the current approach, as you previously spotted, you cannot somehow control dependency constraints.

We have some customization points to specify how to build the particular types, but they are relatively simple and don't support those complex rules.

What would be the best approach to generate an instance of this kind of structure in AutoFixture ? I mean, I will obviously write a customization, but what would you put in there ?

• Would you generate random ints and loop until one of them is a prime (or execute a prime generating algorithm, of course) ? That could be acceptable for this kind of constraint, but if the constraint was more tricky to comply to, that would fast become costly.

• Would you provide a finite list of some acceptable values ?

Well, unfortunately I don't see a silver bullet here and approach depends on situation. If you don't rely on value to be too random, or single SUT consumes only 1-2 prime numbers, then it might be fine to hardcode prime numbers and pick from them (we have ElementsBulider<> built-in helper for those cases). On other hand, if you need a large list of prime numbers and you operate with long prime number sequences, then probably it's better to code an algorithm to generate them dynamically.

Moreover, let's say now that I'm trying to create an instance of something that takes several arguments that can theorically be random individually, but that will do some validation in between them (for instance argA can be in this range of values only if argB is true, and argC must comply to different validation rules depending on argA value, or argC.X property has to match with argA.X property, some stuff like that).

What would you do in this case ?

Really a good question and unfortunately AutoFixture doesn't allow to solve it in a nice way out of the box. Usually I'm trying to isolate customizations for each type, so customization for one type controls creation for a single type only. But in my cases types are independent and obviously it will not work well in your case. Also AutoFixture doesn't provide context out of the box, so when you are writing a customization for a particular type, you cannot clearly understand the context you are creating an object (internally called specimen) in.

On top of my head, I'd say that I'd usually recommend the following strategy:

• Try creating customization for each type in a way it controls creation of single object type only.
• If you need to create dependencies with particular constrains, it's better to activate those dependencies in the customization as well. If your dependency is mutable, you can ask AutoFixture to create the dependency for you and later configure it in a way it becomes compatible.

This way you wouldn't contradict the internal architecture too much and it will be clear how it works. Of course, potentially this way is very verbose.

If cases with complex constrains are not that common, the existing capabilities might be enough for you. But if your domain model is really full of such cases, frankly AutoFixture might be not the best tool for you. Probably, there are better tools on the market which allows to solve such problems in a most elegant way. Of course, it worth mentioning that AutoFixture is very flexible and you can override almost everything, so you can always create your own DSL on top of AutoFixture core... But you should evaluate which way is cheaper to you 😉

Let's also ask @ploeh for his thoughts. Usually Mark's answers are deep and he tries to find the root cause first, rather than solve the consequences 😅

If you have more questions - please ask! I'll be always welcome to answer them.

P.S. FWIW, I decided to provide you with a sample, where I tried to play with AutoFixture and solve a similar problem (I tried to keep it simple and it might not entirety work in your case):

        if (number <= 1) return false;
        if (number == 2) return true;
        if (number % 2 == 0) return false;

        var boundary = (int) Math.Floor(Math.Sqrt(number));

        for (int i = 3; i <= boundary; i += 2)
        {
            if (number % i == 0) return false;
        }

        return true;
    }
}

public class DepA
{
    public int Value { get; set; }
}

public class DepB
{
    public int PrimeNumber { get; }
    public int AnyOtherValue { get; }

    public DepB(int primeNumber, int anyOtherValue)
    {
        if (!Util.IsPrime(primeNumber))
            throw new ArgumentOutOfRangeException(nameof(primeNumber), primeNumber, "Number is not prime.");

        PrimeNumber = primeNumber;
        AnyOtherValue = anyOtherValue;
    }
}

public class DepC
{
    public DepA DepA { get; }
    public DepB DepB { get; }

    public DepC(DepA depA, DepB depB)
    {
        if (depB.PrimeNumber < depA.Value)
            throw new ArgumentException("Second should be larger than first.");

        DepA = depA;
        DepB = depB;
    }

    public int GetPrimeNumber() => DepB.PrimeNumber;
}

public class Issue1067
{
    [Theory, CustomAutoData]
    public void ShouldReturnPrimeNumberFromDepB(DepC sut)
    {
        var result = sut.GetPrimeNumber();

        Assert.Equal(sut.DepB.PrimeNumber, result);
    }
}

public class CustomAutoData : AutoDataAttribute
{
    public CustomAutoData() : base(() =>
    {
        var fixture = new Fixture();

        // Add prime numbers generator, returning numbers from the predefined list
        fixture.Customizations.Add(new ElementsBuilder<PrimeNumber>(2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41));

        // Customize DepB to pass prime numbers only to ctor
        fixture.Customize<DepB>(c => c.FromFactory((PrimeNumber pn, int anyNumber) => new DepB(pn, anyNumber)));

        // Customize DepC, so that depA.Value is always less than depB.PrimeNumber
        fixture.Customize<DepC>(c => c.FromFactory((DepA depA, DepB depB, byte diff) =>
        {
            depA.Value = depB.PrimeNumber - diff;
            return new DepC(depA, depB);
        }));

        return fixture;
    })
    {
    }
}

/// <summary>
/// A helper type to represent a prime number, so that you can resolve prime numbers 
/// </summary>
public readonly struct PrimeNumber
{
    public int Value { get; }

    public PrimeNumber(int value)
    {
        Value = value;
    }

    public static implicit operator int(PrimeNumber prime) => prime.Value;
    public static implicit operator PrimeNumber(int value) => new PrimeNumber(value);
}

Hi @zvirja

Wow, thanks for the detailed answer, it's really interesting. I'll have to do some tests and estimate what it's worth doing or not, but all in all this is great.

I don't think I have so many dependencies to handle, so your approach might be a good way to go. Of course, if @ploeh happens to have something more to add, I would be honored 👌

Thanks again, keep up the good work!

My experience with both AutoFixture and property-based testing is that there's essentially two ways to address issues like these:

• Filtering
• Algorithmic creation

(As I'm writing, my intuition suggests that these could be _catamorphisms_ and _anamorphisms_, respectively, but I'll have to think about this some more, so this aside is mostly a note to myself.)

If _most_ randomly generated values would fit whatever constraints one must fit within, then using an existing generator, but throwing away the occasional unsuitable value, could be the easiest way to address the issue.

If, on the other hand, a filter would mean throwing away most random data, you'd instead have to come up with an algorithm that, perhaps based on random seed values, will generate values that fit the constraints in question.

Some years ago, I gave a talk showing some simple examples of both approaches in the context of FsCheck. This presentation is actually an evolution of a talk that took the same approach, but just with AutoFixture instead. Unfortunately, no recording exists of that talk.

One could address the prime-number requirement in both ways.

The filter approach would be to generate unconstrained numbers, and then throw numbers away until you get one that is, indeed, a prime.

The algorithmic approach would be to use an algorithm like a prime sieve to generate a prime number. This isn't random, though, so one might want to figure out how to randomise it.

The overall question of how to deal with constrained values in AutoFixture came up almost immediately once other people started looking at the library, and I wrote an article back then that I still refer to: http://blog.ploeh.dk/2009/05/01/DealingWithConstrainedInput

Regarding the question of multiple values that relate to each other, I don't wish to give any general guidance. Those sorts of questions are often XY problems. In many cases, once I understand the specifics, an alternative design could solve problems with not only AutoFixture, but also the production code base itself.

Even in the presence of XY problems, though, there's still going to be a situations where this may be a legitimate concern, but I'd rather deal with those on case-by-case basis, as, in my experience, they're rare.

So if you have a specific example of this, I may be able to help, but I don't think I can meaningfully answer the general question.

@ploeh Thanks a lot for this answer, that confirms the approaches I was considering (and you made me curious about cata- and anamorphisms 😃).
I totally agree that interdependent values are mostly an XY problem (at least in my case), the thing is that when working on legacy (untested 😢) code, dealing with those values was a good start to write some tests anyway, until we get the time to refactor this properly.

Anyway, both your answers address the problem quite nicely, I guess I'm good to go from there.
Thanks!

BTW, I forgot to mention that I only meant my answer as an addition to @zvirja's. It's already a good answer there 👍

I didn't take it any other way 😄