Nbench: The specified framework version '2.1' could not be parsed

@HIPERCUBE this might be an issue with your environment - but let me double check.

Do you have a global.json or anything in your project that specifies a runtime version?

Can confirm problems running NBench with dotnet core 2.1

The specified framework version '2.1' could not be parsed
The specified framework 'Microsoft.NETCore.App', version '2.1' was not found.
  - Check application dependencies and target a framework version installed at:
      C:\Program Files\dotnet\
  - Installing .NET Core prerequisites might help resolve this problem:
      http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
  - The .NET Core framework and SDK can be installed from:
      https://aka.ms/dotnet-download
  - The following versions are installed:
      2.0.6 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.0.7 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.0.9 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.1.1 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.1.2 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.1.3-servicing-26724-03 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.1.4 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
      2.1.5 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]

I have no gloabl.json file and other 2.1 projects are running fine. I also installed the 2.1 runtime to double check if there is a version mismatch but it's still not working.

If it's any help, I found adding <RuntimeFrameworkVersion>2.1.5</RuntimeFrameworkVersion> to my .csproj fixed the issue (credit goes to @Rene-Sackers for the solution they listed here: https://github.com/dotnet/cli/issues/10111#issuecomment-429014700).

This appears to be an issue with the dotnet CLI, as far as I can tell - looking through some of the linked issues et al. The errors written out here aren't from NBench - they're from the dotnet process that dotnet-nbench spawns to execute the test runner.

I get the feeling that the root cause of the issue might be the way we're calling the dotnet process, using this DotNetMuxer class we imported from ASP.NET Core:

https://github.com/petabridge/NBench/blob/557f2fbca250a4a45636f5e4b41b58b8440b33f2/src/NBench.Runner.DotNetCli/DotNetMuxer.cs#L1-L74

I created a reproduction project: https://github.com/Aaronontheweb/NBenchNetCore2.1Repro

When I call dotnet build and then dotnet nbench on the sample project in there, I get the following output:

λ  dotnet nbench
Building for framework netcoreapp2.1...
  NBenchRepro -> C:\Repositories\NBenchNetCore2.1Repro\NBenchRepro\bin\Release\netcoreapp2.1\NBenchRepro.dll
Running .NET Core 2.1.0 tests for framework netcoreapp2.1...
OutputDir C:\Repositories\NBenchNetCore2.1Repro\NBenchRepro\PerfResults
Executing Benchmarks in C:\Repositories\NBenchNetCore2.1Repro\NBenchRepro\bin\Release\netcoreapp2.1\NBenchRepro.dll
------------ STARTING NBenchRepro.CounterPerfSpecs+Benchmark ----------
--------------- BEGIN WARMUP ---------------
Elapsed: 00:00:00.1657419
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 153,490,119.28 , ns / operations: 6.52
--------------- END WARMUP ---------------

--------------- BEGIN WARMUP ---------------
Elapsed: 00:00:00.1663968
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 152,886,017.04 , ns / operations: 6.54
--------------- END WARMUP ---------------

--------------- BEGIN WARMUP ---------------
Elapsed: 00:00:00.1670527
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 152,285,739.77 , ns / operations: 6.57
--------------- END WARMUP ---------------

--------------- BEGIN RUN ---------------
Elapsed: 00:00:00.1659459
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 153,301,431.37 , ns / operations: 6.52
--------------- END RUN ---------------

--------------- BEGIN RUN ---------------
Elapsed: 00:00:00.1654938
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 153,720,223.96 , ns / operations: 6.51
--------------- END RUN ---------------

--------------- BEGIN RUN ---------------
Elapsed: 00:00:00.1651669
[Counter] TestCounter - operations: 25,439,744.00 ,operations: /s 154,024,468.58 , ns / operations: 6.49
--------------- END RUN ---------------

--------------- RESULTS: NBenchRepro.CounterPerfSpecs+Benchmark ---------------
Test to ensure that a minimal throughput test can be rapidly executed.
--------------- DATA ---------------
[Counter] TestCounter: Max: 25,439,744.00 operations, Average: 25,439,744.00 operations, Min: 25,439,744.00 operations, StdDev: 0.00 operations
[Counter] TestCounter: Max / s: 154,024,468.58 operations, Average / s: 153,682,041.30 operations, Min / s: 153,301,431.37 operations, StdDev / s: 363,027.74 operations

--------------- ASSERTIONS ---------------
[PASS] Expected [Counter] TestCounter to must be greater than 10,000,000.00 operations; actual value was 153,682,041.30 operations.

------------ FINISHED NBenchRepro.CounterPerfSpecs+Benchmark ----------

Using NBench 1.2.2 and dotnet --version 2.1.300 in this case.

So this looks very much like an environment issue, as has often been the case with similar problems with dotnet xunit in the past. If you could provide us with some more information on what your environment looks like or a reproduction spec, I'll reopen this issue and keep chipping away at it.

In the meantime, you could try a work-around by specifying an explicit runtime version per @C-Babbage's suggestion.

Following up on this some - was able to reproduce this issue myself while working on https://github.com/petabridge/Petabridge.Tracing.Zipkin/pull/68

I reached the same conclusion as the Entity Framework and XUnit teams: this is an issue with the CLI itself and the work-around appears to be specifying the RuntimeFrameworkVersion inside your test projects. That gives the CLI the necessary information it needs to correctly resolve your .NET Core runtime.

Hopefully this is improved in the future, but this error occurs after dotnet-nbench spins up a new dotnet process to run the specs - it's a CLI issue.