Sinon: Spy `returnValue` does not work with generators

Sinon does not support ES6 features yet. Work on this has not yet started, so I don't know when this will work.

@mantoni, right, this was more of a feature request :) do you want to leave it open, for instance, with some kind of tag ("es6"?) so that when that is added to the roadmap it is easier to re-evaluate what needs to be done?

This would be a great feature to have.

@ruimarinho i found this https://github.com/ingameio/SinonES6.JS
replacing require("sinon") with require("sinon-es6") seems to make your tests pass

--
edit by @fatso83 on June 22nd 2017:
This does not seem to be true. I manually tested this. It fails all the same, which makes total sense as sinon-es6 _only implements its support for generators for mocking_.

Is anyone working on this?

@emgeee not that I am aware of.

+1 @ruimarinho sinon.js with es6 features would be awesome to have!

@ingameio care to make a PR with your changes?

@fatso83 thanks for the suggestion!
I'll do the PR soon ;)

I'm having problems when running buster-test against the packaged version. I did some debugging but I'm short of time this days so I couldn't solve the issue.
If someone want to help I could push the changes to my fork, just tell me and I will do it.

@gaguirre That is great news. Maximillian just converted all the tests to Mocha, and did some changes in the test setup while at it, so maybe things work out if you pull in the latest changes from master?

@gaguirre In case someone should stumble over this thread, I think it could be an idea to push the changes to your fork anyhow so that a passerby might have a look at it.

@fatso83 I pushed a first version to underscope/sinon.
The main issue occurs when running the tests with an engine that not support es6 (phantomjs in this case): it fails when parsing the code so I'm using eval() as a workaround since can be surrounded by a try/catch. I think this is not acceptable but it's a starting point.

Right now I'm trying to wrap the generator call in another file and require it dynamically, but it's not working, I guess because browserify it's bundling even the dynamically required files. I'm wondering if some files could be excluded when running npm run test-headless.

What do you think could be the best solution?

Thanks for the feedback, @gaguirre . So the basic issue is that we need some way of avoiding parsing if the engine does not support it. That is a bit harder than simple feature detection, like we do with WebWorkers, etc.

Just delegating the actual check is as simple as if(require('generator-support')){ ... }, but this does not fix the actual parsing problem.

Any ideas on how to cleanly deal with this, @mantoni and @mroderick?

P.S. I am away on offline-holiday in a few hours, so won't be able to read the answers until some time after Easter.

I can't tell if this level of generator stubbing actually works (particularly in coroutine-oriented scenarios) - it seems some emulation of async behavior might be in order. But I do think there's a way around the parsing issue.

Suggestion:

Extract the generator code into a separate file and require() that file only on demand when wrapping a generator function, i.e.:

?/es6-support.js:

"use strict";

exports.getGeneratorWrapper = function(method, ctx, args) {
    return function* () {
        return mockObject.invokeMethod(method, ctx, args);
    }
}

lib/sinon/mock.js->expects...:

var wrapper = function () {
    return mockObject.invokeMethod(method, this, arguments);
});

if (/^function\s*\*/.test(method.toString())) {
    wrapper = require('es6-support').getGeneratorWrapper(method, this, arguments);
}

wrapMethod(this.object, method, wrapper);

(Also do similar with the unit tests, and ensure tracking of code coverage doesn't cause the 'es6-support' file to be automatically included.)

Alternative suggestion:

Is ES5 compatibility still going to be a worthwile objective by the time sinon 2.0 is ready for release? Perhaps it's time to tell the few people still supporting legacy ES5-only environments without the user of transpilers that they'll be left behind on 1.x.

@evan-king That conditional require is not a solution, as it will break in our browser builds. As conditional requires defeats static analysis of the dependency graph, tools such as WebPack, Rollup and Browserify will be unable to deal with it. It's either totally in or totally out.

And yes, ES5 compatibility is a thing. ES6 generator support is shoddy at best, and forcing people to use additional tooling to use Sinon for their projects will raise the bar for doing testing. And to many, that threshold is high enough as it is. Being able to simple include a script tag to get it working is an important feature IMHO. We might break ES5 compatibility at some time, but it is not on our roadmap and has not even been discussed for Sinon 3. Sinon 2 has in effect been out for quite some time; there are just some minor nuisances stopping us from doing an official release.

There are basically two ways of getting ES6 compatibility without breaking existing ES5 runtime compatibility that I can come up ith, and I am not really sure about the last one (haven't testet it):

Conditional evaluation of modules

This is a hack, but is fairly simple. What it means is that we can rely on static asset inlining and ES6 feature testing to decide whether to evaluate the required module. This will ensure ES5 compatibility (Sinon will only be patched if the runtime supports the syntax), there is no need for additional tooling in the form of Babel on neither the library maker side nor the client user, and testing ES6 will break ES5 browsers that would have failed in any case.

Assuming something like brfs is in place the code would look something like this:

_runtime-features.js_

var features = {};
try { 
    new Function("var foo = function* foo(){ }") ;
    features.generatorSupport = true; 
} 
catch(err){ features.generatorSupport = false; }
module.exports = features;

_es6-generator-wrapper.js_

return function* () {
    return mockObject.invokeMethod(method, ctx, args);
}

_es6.js_

var features = require('./runtime-features');

if( features.generatorSupport ) {
    var code = fs.readFileSync('./es6-generator-wrapper.js');
    module.exports.getGeneratorWrapper = new Function("mockObject", "method", "ctx", "args", code);
} else {
    module.exports.getGeneratorWrapper = function() { throw new TypeError("You tried using a generator function in a runtime only capable of running ES5 code"); }
}

Babelifying Sinon

This is the one I am not sure about as I haven't actually tried. If we transpile the build through Babel to ES5 we can write ES6 code all over the place, avoid jumping through hoops such as I did above, and we can still use the same kinds of checks for generators. They will just be implemented using ES5 constructs. Of course, testing ES6 in ES5 browsers will still fail. This has the same upside as the previous one on the client side, but we might hinder contributions as ES2015 knowledge on stuff such as yield, async, function*() is far from reaching a wide audience.

+1

@rpavlovs, there is really no point in adding +1's to the thread. The GitHub UI has a "add reaction" button on top of each comment if you need to express your emotions. A +1 will do nothing. A pull request that implements one of the suggestions above (or something more clever), on the other hand, has a far better chance of fixing this issue 😄

I would like input on how API support for generators would look, as after using a couple of hours on this, I am still not really sure what people would like to see.

To start fleshing this out, I have created a new branch that holds @ingameio's modifications to the mock api, while at the same time not breaking ES5 compatibility (using the hack mentioned above).

What irks me a bit is that I really can't tell how to test the original changes by Ingameio, as none of the test examples work - not even the example in the fork is complete, and I cannot get stuff to break pre/post changes.

Generators are simple things: gentle synchronous beings that remember their past. So please don't riddle any examples with co and other things that are unrelated, as it makes it harder to see what is wanted/doesn't work. The top example for instance is quite complicated, and it also seems to mistake what yield does, as it expects the return value of the "yield expression" to be the same as the "yielded value". The result of the yield is the value passed into the generator's next() (MDN)

I do realize the examples using co probably only uses it to be able to use yield directly in the Mocha test, but just wrap your example in an IIFE or some other way of achieving the same for the sake of more clarity.

This is a simple test of how generators support (works in today's Sinon):

require("should");

var sinon = require("../sinon");

var foo = {
    bar: function () {
        return "bar";
    },
    biz: function *() {
        return "biz";
    }
};

describe("generator support", function () {
    it('should work with generator functions',  function(){
        var spy = sinon.spy(foo, 'biz');

        var iterator = foo.biz();
        var result = iterator.next();

        result.value.should.equal('biz');
        result.done.should.equal(true);
        spy.firstCall.returnValue.should.be.an.Object();
        spy.firstCall.returnValue.next.should.be.a.Function();
    });
});

Now, what API extensions would we like?
From the original test, I assume we would like to see something like

foo.biz.firstGeneratedValue.should.equal('biz');
or
foo.biz.generatedValue[0].should.equal('biz');
?

cc @ruimarinho

I am closing this issue, as the original test had an error, and I cannot see any issues with the generator handling in Sinon.

Please join the discussion on how an API for dealing with generators (and its associated iterators) would look in issue #1467