Autofixture: Proposal for GenerationDepthBehavior.

Created on 1 May 2018  ·  12Comments  ·  Source: AutoFixture/AutoFixture

Here is a generation Depth limiter behavior proposal.

fixture.Behaviors.Add(new GenerationDepthBehavior(2));

Had to re-implement ComposeIfMultiple for test because it's internal.

```c#
public class GenerationDepthBehavior : ISpecimenBuilderTransformation
{
private const int DefaultGenerationDepth = 1;
private readonly int generationDepth;

public GenerationDepthBehavior() : this(DefaultGenerationDepth)
{
}

public GenerationDepthBehavior(int generationDepth)
{
    if (generationDepth < 1)
        throw new ArgumentOutOfRangeException(nameof(generationDepth), "Generation depth must be greater than 0.");

    this.generationDepth = generationDepth;
}


public ISpecimenBuilderNode Transform(ISpecimenBuilder builder)
{
    if (builder == null) throw new ArgumentNullException(nameof(builder));

    return new GenerationDepthGuard(builder, new GenerationDepthHandler(), this.generationDepth);
}

}

public interface IGenerationDepthHandler
{

object HandleGenerationDepthLimitRequest(object request, IEnumerable<object> recordedRequests, int depth);

}

public class DepthSeededRequest : SeededRequest
{
public int Depth { get; }
public DepthSeededRequest(object request, object seed, int depth) : base(request, seed)
{
Depth = depth;
}
}

public class GenerationDepthGuard : ISpecimenBuilderNode
{
private readonly ThreadLocal> requestsByThread
= new ThreadLocal>(() => new Stack());

private Stack<DepthSeededRequest> GetMonitoredRequestsForCurrentThread() => this.requestsByThread.Value;


public GenerationDepthGuard(ISpecimenBuilder builder)
    : this(builder, EqualityComparer<object>.Default)
{
}

public GenerationDepthGuard(
    ISpecimenBuilder builder,
    IGenerationDepthHandler depthHandler)
    : this(
        builder,
        depthHandler,
        EqualityComparer<object>.Default,
        1)
{
}

public GenerationDepthGuard(
    ISpecimenBuilder builder,
    IGenerationDepthHandler depthHandler,
    int generationDepth)
    : this(
        builder,
        depthHandler,
        EqualityComparer<object>.Default,
        generationDepth)
{
}


public GenerationDepthGuard(ISpecimenBuilder builder, IEqualityComparer comparer)
{
    this.Builder = builder ?? throw new ArgumentNullException(nameof(builder));
    this.Comparer = comparer ?? throw new ArgumentNullException(nameof(comparer));
    this.GenerationDepth = 1;
}


public GenerationDepthGuard(
    ISpecimenBuilder builder,
    IGenerationDepthHandler depthHandler,
    IEqualityComparer comparer)
    : this(
    builder,
    depthHandler,
    comparer,
    1)
{
}

public GenerationDepthGuard(
    ISpecimenBuilder builder,
    IGenerationDepthHandler depthHandler,
    IEqualityComparer comparer,
    int generationDepth)
{
    if (builder == null) throw new ArgumentNullException(nameof(builder));
    if (depthHandler == null) throw new ArgumentNullException(nameof(depthHandler));
    if (comparer == null) throw new ArgumentNullException(nameof(comparer));
    if (generationDepth < 1)
        throw new ArgumentOutOfRangeException(nameof(generationDepth), "Generation depth must be greater than 0.");

    this.Builder = builder;
    this.GenerationDepthHandler = depthHandler;
    this.Comparer = comparer;
    this.GenerationDepth = generationDepth;
}


public ISpecimenBuilder Builder { get; }

public IGenerationDepthHandler GenerationDepthHandler { get; }

public int GenerationDepth { get; }

public int CurrentDepth { get; }

public IEqualityComparer Comparer { get; }

protected IEnumerable RecordedRequests => this.GetMonitoredRequestsForCurrentThread();

public virtual object HandleGenerationDepthLimitRequest(object request, int currentDepth)
{
    return this.GenerationDepthHandler.HandleGenerationDepthLimitRequest(
        request,
        this.GetMonitoredRequestsForCurrentThread(), currentDepth);
}

public object Create(object request, ISpecimenContext context)
{
    if (request is SeededRequest)
    {
        int currentDepth = -1;

        var requestsForCurrentThread = this.GetMonitoredRequestsForCurrentThread();

        if (requestsForCurrentThread.Count > 0)
        {
            currentDepth = requestsForCurrentThread.Max(x => x.Depth) + 1;
        }

        DepthSeededRequest depthRequest = new DepthSeededRequest(((SeededRequest)request).Request, ((SeededRequest)request).Seed, currentDepth);

        if (depthRequest.Depth >= this.GenerationDepth)
        {
            return HandleGenerationDepthLimitRequest(request, depthRequest.Depth);
        }

        requestsForCurrentThread.Push(depthRequest);
        try
        {
            return this.Builder.Create(request, context);
        }
        finally
        {
            requestsForCurrentThread.Pop();
        }
    }
    else
    {
        return this.Builder.Create(request, context);
    }
}

public virtual ISpecimenBuilderNode Compose(
    IEnumerable<ISpecimenBuilder> builders)
{
    var composedBuilder = ComposeIfMultiple(
        builders);
    return new GenerationDepthGuard(
        composedBuilder,
        this.GenerationDepthHandler,
        this.Comparer,
        this.GenerationDepth);
}

internal static ISpecimenBuilder ComposeIfMultiple(IEnumerable<ISpecimenBuilder> builders)
{
    ISpecimenBuilder singleItem = null;
    List<ISpecimenBuilder> multipleItems = null;
    bool hasItems = false;

    using (var enumerator = builders.GetEnumerator())
    {
        if (enumerator.MoveNext())
        {
            singleItem = enumerator.Current;
            hasItems = true;

            while (enumerator.MoveNext())
            {
                if (multipleItems == null)
                {
                    multipleItems = new List<ISpecimenBuilder> { singleItem };
                }

                multipleItems.Add(enumerator.Current);
            }
        }
    }

    if (!hasItems)
    {
        return new CompositeSpecimenBuilder();
    }

    if (multipleItems == null)
    {
        return singleItem;
    }

    return new CompositeSpecimenBuilder(multipleItems);
}

public virtual IEnumerator<ISpecimenBuilder> GetEnumerator()
{
    yield return this.Builder;
}

IEnumerator IEnumerable.GetEnumerator()
{
    return this.GetEnumerator();
}

}

public class GenerationDepthHandler : IGenerationDepthHandler
{
public object HandleGenerationDepthLimitRequest(
object request,
IEnumerable recordedRequests, int depth)
{
return new OmitSpecimen();
}
}
```

question

Most helpful comment

@malylemire1 Would you be fine if we postpone this for now and implement it later only if we see a lot of requests for this feature?

I'm sorry we cannot include every desired feature to the core 😟 It's a bit hard to keep the balance between the provided feature and and the amount of code we maintain here. It looks like it will not be a big problem for you to implement this ad-hoc feature in your solution, so you are not blocked.

Thanks again for your proposal!

All 12 comments

Hello, thanks for sharing the idea! :+1:

Could you please describe the reasoning behind this feature? Do you have any particular scenario where you found it useful?

Hi, I use it mainly for ORM entites or external api classes where depth can propagate indefinitely but is not recursive.
I use AutoFixture for Unit Tests, Integration Tests and Functional Tests. Unit test for core enterprise framework functionalities. Integration tests with tests for complex units of work with repository pattern and access to a test database. Functional test with access to an entire application.

I have put it directly on AutoMoqDataAttribute

```c#
public class AutoMoqDataAttribute : AutoDataAttribute
{
///


/// Default constructor.
///

/// Default to 0 for unlimited generation depth.
public AutoMoqDataAttribute(int generationDepth = 0)
: base(() =>
{
IFixture fixture = new Fixture().Customize(new AutoMoqCustomization());

        fixture.Behaviors.OfType<ThrowingRecursionBehavior>().ToList()
                .ForEach(b => fixture.Behaviors.Remove(b));

        fixture.Behaviors.Add(new OmitOnRecursionBehavior());

        if (generationDepth > 0)
        {
            fixture.Behaviors.Add(new GenerationDepthBehavior(generationDepth));
        }

        return fixture;
    })
{
}

}
```

I would also suggest to make ComposeIfMultiple public. It would make custom behavior implementation easier.

I do understand what the behavior does, the main question is the purpose and scenarios.

Hi, I use it mainly for ORM entites or external api classes where depth can propagate indefinitely but is not recursive.

So, as far as I understand, the main goal you are aiming is the performance. From the functional perspective everything works fine and you simply would like to save a few CPU ticks.

It's a bit hard to judge whether we would like to ship with this feature out of the box or not. I do understand that you might indeed have type model with very deep nesting and there the performance win is significant. But there are also a few drawbacks with it:

  • it breaks the transparency of your unit tests. Now you need to know the level of the deepest accessed object during the test. If you later change implementation, tests might start to break due to default values and you will probably need to update the level.
  • it increases test maintenance, as now you need to track a fact that graph might be not initialized entirely.

It's completely fine that in your team you require this feature and you agree to use it. However, I'd argue to give it a common usage as it complicates things a lot.

There are a lot of potential scenarios and AutoFixture tries to ship the most common features only. The more advanced bits are covered by the extensibility capabilities. In this particular case I'd say that feature is not that common to make it a part of the product. Another point is implementation is very tiny (see below), so it should not be a problem to implement it in your test project locally.

I would also suggest to make ComposeIfMultiple public. It would make custom behavior implementation easier.

It has been already discussed #657.


By the way, implementation might be simplified a bit (if I'm not missing something):
```c#
public class GenerationDepthBehavior: ISpecimenBuilderTransformation
{
public int Depth { get; }

public GenerationDepthBehavior(int depth)
{
    Depth = depth;
}

public ISpecimenBuilderNode Transform(ISpecimenBuilder builder)
{
    return new RecursionGuard(builder, new OmitOnRecursionHandler(), new IsSeededRequestComparer(), Depth);
}

private class IsSeededRequestComparer : IEqualityComparer
{
    bool IEqualityComparer.Equals(object x, object y)
    {
        return x is SeededRequest && y is SeededRequest;
    }

    int IEqualityComparer.GetHashCode(object obj)
    {
        return obj is SeededRequest ? 0 : EqualityComparer<object>.Default.GetHashCode(obj);
    }
}

}
```


@moodmosaic Could you please also share your opinion? 😉

The example shown in https://github.com/AutoFixture/AutoFixture/issues/1032#issuecomment-385724150 is the preferred way of using this ad-hoc behavior. At this point, I don't think we should be changing the built-in algorithm.


What we could consider, however, is to

  • use a splittable PRNG (the current one isn't splittable)
  • add randomness tests (follow https://github.com/hedgehogqa/haskell-hedgehog/issues/125, for example)
  • benchmarks

and then, we could consider taking on further optimizations, like this one.

@moodmosaic It seems the suggestion is not to change the default behavior, but to make GenerationDepthBehavior a part of the AutoFixture library (so you could enable it on demand). So what is your opinion regarding this? :wink:

@zvirja It's not actually for performance reasons. The problem with indefinite propagation is failed test with out of memory exception.

@zvirja, @malylemire1, unless this ad-hoc behavior addresses a wide range of scenarios, I don't think it has to be built-in.


I can't spot an MCVE, but AFAICT, the issues are coming from automated-tests that aren't actually unit tests (e.g. boundary/integration tests).

AutoFixture wasn't primarily designed to deal with integration tests, so some ad-hoc behaviors (like the one shown here) are warranted :+1: I don't think it has to be part of the core library, though.

@malylemire1 Would you be fine if we postpone this for now and implement it later only if we see a lot of requests for this feature?

I'm sorry we cannot include every desired feature to the core 😟 It's a bit hard to keep the balance between the provided feature and and the amount of code we maintain here. It looks like it will not be a big problem for you to implement this ad-hoc feature in your solution, so you are not blocked.

Thanks again for your proposal!

Closing this issue as a part of the cleanup. Thanks again for sharing the idea! 😉

@malylemire1 I 100% understand the use case.
Is this something you can make available in a nuget package?

Was this page helpful?
0 / 5 - 0 ratings

Related issues

josh-degraw picture josh-degraw  ·  4Comments

zvirja picture zvirja  ·  3Comments

Ridermansb picture Ridermansb  ·  4Comments

mjfreelancing picture mjfreelancing  ·  4Comments

ploeh picture ploeh  ·  3Comments