Protractor: Junit Reporter

My 2 cents here: if you use Grunt as your build tool, you can run your Protractor tests using grunt-jasmine-node and configure it to get JUnit-like reports!

How do to this in your configuration file in an only slightly hacky way - I've tested this out with Protractor's own test suite:

npm install jasmine-reporters

Your config file:

require('jasmine-reporters');

jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(
    'outputdir/', true, true));

// The main suite of Protractor tests.
exports.config = {
  seleniumServerJar: './selenium/selenium-server-standalone-2.35.0.jar',
  chromeDriver: './selenium/chromedriver',

  seleniumAddress: 'http://localhost:4444/wd/hub',

  // Spec patterns are relative to this directory.
  specs: [
    '*_spec.js'
  ],

  capabilities: {
    'browserName': 'chrome'
  },

  baseUrl: 'http://localhost:8000',
};

Further examples are now shown in https://github.com/angular/protractor/blob/master/spec/junitOutputConf.js

Does it accept the relative path or absolute of directory.Absolute path is working while relative path not. I have tried options like ../Reports but its of no use
jasmine.getEnv().addReporter(
new jasmine.JUnitXmlReporter('xmloutput', true, true));
},

explored and working , ignore my last comment

I wasn't smart enough so I expected the folder to be created but in fact, the folder has to be created before hand.

@juliemr How to use it with two Protractor instances running in parallel, e.g. via grunt-concurrent? It's needed so that we can run tests in Firefox & Chrome at the same time but it'll make one XML replace the other which will confuse Jenkins.

Maybe Protractor should know how to run in Parallel for reasons like that, similarly to what Karma does?

@mzgol Have you tried making each target running as a concurrent task output a different junit xml report based on the platform? You could just create a new output folder based on the target name.

You may also want to look into multi-configuration targets on Jenkins which let you run your build steps in parallel each with a clean environment. It also has nifty features like touchstone builds and allows you to throttle concurrency better than grunt.

Does similar config exist for mocha? (as I am not using Jasmine)

@mzgol I tried the suggestion from jvandyke(thank you) and it is working fine for me:

onPrepare: function(){
        require('jasmine-reporters');
        var capsPromise = browser.getCapabilities();
        capsPromise.then(function(caps){
            var browserName = caps.caps_.browserName.toUpperCase();
            var browserVersion = caps.caps_.version;
            var prePendStr = browserName + "-" + browserVersion + "-";
            jasmine.getEnv().addReporter(new
jasmine.JUnitXmlReporter("protractor_output", true, true,prePendStr));
        });
 },

Thanks. Its working for me too:

onPrepare: function(){
require('jasmine-reporters');
var capsPromise = browser.getCapabilities();
capsPromise.then(function(caps){
var browserName = caps.caps_.browserName.toUpperCase();
var browserVersion = caps.caps_.version;
var prePendStr = browserName + "-" + browserVersion + "-";
jasmine.getEnv().addReporter(new
jasmine.JUnitXmlReporter("protractor_output", true, true,prePendStr));
});
},

I have created an html reporter on top of this. Please let me know your comments
https://github.com/jintoppy/protractor-html-screenshot-reporter

It might be an idea to add the folder creation to docs as I tripped up on that too @tracycummins !

@juliemr Tired all options listed in samples. I get this error. Any pointers appreciated

/home/jack/prototype/conf.js:19
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('xmloutput', true, true));
^
TypeError: undefined is not a function
at exports.config.onPrepare (/home/jack/prototype/conf.js:19:33)
at Runner.runFilenamesOrFns_ (/home/usr/lib/node_modules/protractor/lib/runner.js:64:9)
at Runner.runTestPreparers (/home/usr/lib/node_modules/protractor/lib/runner.js:91:8)
at runner.controlFlow.execute.then.jasmineNodeOpts (/home/usr/lib/node_modules/protractor/lib/frameworks/jasmine.js:39:12)
at webdriver.promise.ControlFlow.runInNewFrame_ (/home/usr/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1539:20)
at webdriver.promise.ControlFlow.runEventLoop_ (/home/usr/lib/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver/promise.js:1404:8)
at wrapper as _onTimeout
at Timer.listOnTimeout as ontimeout
==== async task ====
run test preparers
at Object.exports.run (/home/usr/lib/node_modules/protractor/lib/frameworks/jasmine.js:38:24)
at driverprovider_.setupEnv.then.then.then.testResult (/home/usr/lib/node_modules/protractor/lib/runner.js:233:28)
at _fulfilled (/home/usr/lib/node_modules/protractor/node_modules/q/q.js:797:54)
at self.promiseDispatch.done (/home/usr/lib/node_modules/protractor/node_modules/q/q.js:826:30)
at Promise.promise.promiseDispatch (/home/usr/lib/node_modules/protractor/node_modules/q/q.js:759:13)
at /home/usr/lib/node_modules/protractor/node_modules/q/q.js:525:49
at flush (/home/usr/lib/node_modules/protractor/node_modules/q/q.js:108:17)
at process._tickCallback (node.js:423:13)
at Function.Module.runMain (module.js:499:11)

// An example configuration file.
exports.config = {
// The address of a running selenium server.
seleniumAddress: 'http://localhost:4444/wd/hub',
allScriptsTimeout: 30000,
// Capabilities to be passed to the webdriver instance.
capabilities: {
'browserName': 'firefox'
},

// Spec patterns are relative to the current working directly when
// protractor is called.
specs: ['example_spec.js'],

// Options to be passed to Jasmine-node.
onPrepare: function() {
require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('xmloutput', true, true));
},

// Options to be passed to Jasmine-node.
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 30000
}
};

hey @jkmurthy

I ran into the same exact issue. In the latest version of [email protected], they changed the module name. There is no documentation on this, but its there in this file.

https://github.com/larrymyers/jasmine-reporters/blob/master/src/junit_reporter.js

• Usage:
  
  
  
  • jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter(options);
  
  

Still I couldn't make it work as it says module 'jasmineReporters' undefined.

So I switched to the older version of [email protected], it worked perfectly. Make sure you create the directory whichever you are referring as the output in reporter options.

Downgrade jasmine-reporters to 0.4.1 did not work
Tried with
onPrepare: function() {
var folderName = (new Date()).toString().split(' ').splice(1, 4).join(' ');
var mkdirp = require('mkdirp');
var newFolder = "./reports/" + folderName;
require('jasmine-reporters');

            mkdirp(newFolder, function(err) {
              if (err) {
                console.error(err);
              } else {
                jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(newFolder, true, true));
              }
            });
          },

No luck yet

This is definitely an issue for me. Using jasmine-reporters 2.0 I get
D:\HgBuild\TPSOneSearch\Unstable\Tyler.TPS.OneSearch\Tyler.TPS.Web.OneSearch\app
\test\e2e\oneSearchChromeConf.js:29
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter(uri, t
rue, true, ''));
^
TypeError: undefined is not a function.

changing "new jasmine.JUnitXmlReporter" to "jasmineReporters.JUnitXmlReporter" results in jasmineReporters is undefined.

If I revert jasmine-reporters back to 0.4.1 it works splendidly.

jasmine-reporters 2.0 has different API than 1.x; you can't just blindly update; adjust your invocations.

Looking at the comment in junit_reporter.js file for JUnitXmlReporter I have setup my conf file as follows:

onPrepare: function () {
require('jasmine-reporters');
var options = {
savePath: 'some-file-path',
consolidate: true,
useDotNotation: true
};
jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter(options));
},

this should work should it not?
I get the following error "ReferenceError: jasmineReporters is not defined"

/**
* Generates JUnit XML for the given spec run. There are various options
* to control where the results are written, and the default values are
* set to create as few .xml files as possible. It is possible to save a
* single XML file, or an XML file for each top-level describe, or an
* XML file for each describe regardless of nesting.
*
* Usage:
*
* jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter(options);
*
* @param {object} [options]
* @param {string} [savePath] directory to save the files (default: '')
* @param {boolean} [consolidateAll] whether to save all test results in a
* single file (default: true)
* NOTE: if true, {filePrefix} is treated as the full filename (excluding
* extension)
* @param {boolean} [consolidate] whether to save nested describes within the
* same file as their parent (default: true)
* NOTE: true does nothing if consolidateAll is also true.
* NOTE: false also sets consolidateAll to false.
* @param {boolean} [useDotNotation] whether to separate suite names with
* dots instead of spaces, ie "Class.init" not "Class init" (default: true)
* @param {string} [filePrefix] is the string value that is prepended to the
* xml output file (default: junitresults-)
* NOTE: if consolidateAll is true, the default is simply "junitresults" and
* this becomes the actual filename, ie "junitresults.xml"
*/

@john-bridges Can you wrap your code in back-ticks? It's hard to read without it.

@john-bridges Well, no wonder the jasmineReporters variable is undefined since you didn't define it, indeed. You need to assign the module:

var jasmineReporters = require('jasmine-reporters');

@mzgol I am an idiot, works like a charm. Thanks for your help I appreciate it.

The previous version of jasmine-reporters attached the reporters directly to the jasmine object; now it attaches to its own export which is way better.

I see where the confusion came from. ;)

@mzgol I agree, a much better way, thanks again.

[email protected] supports Jasmine 2.x and, as observed / discussed above, has a different syntax and entry point (on its own object vs global jasmine object). That said, it _only_ supports Jasmine 2.x, not Jasmine 1.x which is what Protractor uses. So, while you may be able to get past errors in your protractor config by doing var jasmineReporters = require('jasmine-reporters'); new jasmineReporters.JUnitXmlReporter(/*...*/);, you'll be hard pressed to get any meaningful output because it is waiting for Jasmine 2.x events and reporter interface.

The proper way to get JUnitXmlReporter working with Jasmine 1.x is to slightly modify @juliemr 's original instructions:

npm install jasmine-reporters@~1.0.0

Then follow the rest of her instructions as normal.

I have overhauled the jasmine-reporters README to address the ways [email protected] differs from [email protected]. I also added a specific section about using jasmine-reporters with Protractor since the question comes up a lot.

Thanks @ksraja for pointing out the seriously lacking documentation.

Why does this issue keep getting closed, it is apparent that this is a issue still.

    onPrepare: function() {

        require('jasmine-reporters');
        jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('test-target/', true, true));

        browser.driver.get('http://localhost:9000');
        require('./test/e2e/models/login-page').login();
    },

Throws error:

Running "protractor:e2e" (protractor) task

/Users/212400520/PredixExperience/predix-dashboard/node_modules/jasmine-reporters/src/jasmine.console
_reporter.js:2
    if (! jasmine) {
          ^
ReferenceError: jasmine is not defined
    at /Users/212400520/PredixExperience/predix-dashboard/node_modules/jasmine-reporters/src/jasmine.
console_reporter.js:2:11
    at Object.<anonymous> (/Users/212400520/PredixExperience/predix-dashboard/node_modules/jasmine-re
porters/src/jasmine.console_reporter.js:144:3)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/Users/212400520/PredixExperience/predix-dashboard/node_modules/jasmine-re
porters/src/load_reporters.js:1:63)
    at Module._compile (module.js:456:26)
>> 
Fatal error: protractor exited with code: 8

@jonniespratley I found that npm install jasmine-reporters installs 1.0.1 version by default.
So, make sure you have installed latest version of jasmine-reporters. For example

npm install jasmine-reporters@~2.0.3 --save-dev

After that, I updated protractor configuration as follows:

var jasmineReporters = require('jasmine-reporters');
exports.config = {
    framework: 'jasmine2',
    //...  
    //Other configurations...
    //...  
    onPrepare: function () {
        jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter({
            savePath: '..',
            consolidateAll: false
        }));
    },
    jasmineNodeOpts: {
        showColors: true,
        defaultTimeoutInterval: 60000,
        print: function() {}
    }
};

Now jasmine-reporters is working fine to me.

In protractors' documentation for upgrading to jasmine2 framework, in the reporters section it would be nice if @luixaviles answer were documented. I can't imagine I'm the only one that had a difficult time figuring out how to get junit reports for jasmine2. Granted, once I found this and reread the current documentation it is clearer, but originally I mistakenly read it to mean jasmine-reporters didn't work (at all) with jasmine2. Having the above answer, or a link to jasmine-reporters' readme would make things a bit easier.

tried the jasmine reports
var jasmineReporters = require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter('../TestReports/test', true, true));

it ran good, but no xml file is been created

i tried to got to https://github.com/angular/protractor/blob/master/spec/junitOutputConf.js, but gives 404

@spatchamatla Your example is using a weird combination of [email protected] syntax (ordered arguments) with what looks like the [email protected] reporter (jasmineReporters.JUnitXmlReporter vs jasmine.JUnitXmlReporter).

Try using the correct syntax to create the reporter and see if you get further.

Modified from the example in the jasmine-reporters README:

var junitReporter = new jasmineReporters.JUnitXmlReporter({
    savePath: '../TestReports/test'
});
jasmine.getEnv().addReporter(junitReporter);

Make sure that the folder '../TestReports/test' exists as I think it fails
silently too.

On Wed, Apr 15, 2015 at 4:33 PM, Ben Loveridge [email protected]
wrote:

@spatchamatla https://github.com/spatchamatla Your example is using a
weird combination of [email protected] syntax (ordered arguments)
with what looks like the [email protected] reporter
(jasmineReporters.JUnitXmlReporter vs jasmine.JUnitXmlReporter).

Try using the correct syntax to create the reporter and see if you get
further.

Modified from the example in the jasmine-reporters README:

var junitReporter = new jasmineReporters.JUnitXmlReporter({
savePath: '../TestReports/test'
});
jasmine.getEnv().addReporter(junitReporter);

—
Reply to this email directly or view it on GitHub
https://github.com/angular/protractor/issues/60#issuecomment-93451490.

Never a bad idea, though jasmine-reporters will use https://github.com/substack/node-mkdirp to try and create the directory.

@bloveridge @spenoir , i downgraded to [email protected] and used the below

   //For junit output reports
        var reporters = require('jasmine-reporters');
        var capsPromise = browser.getCapabilities();
        capsPromise.then(function (caps) {
            var browserName = caps.caps_.browserName.toUpperCase();
            var browserVersion = caps.caps_.version;
            var prePendStr = browserName + "-" + browserVersion + "-";
            jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('Protractor/TestReports', true, true, prePendStr));
        });

it started working, the only down side is its creating the folder in home directory rather than in project folder. When i tried using the project folder path, it still says

Warning: writing junit report failed for '../TestReports', 'CHROME-41.0.2272.118-TestImpersonationPage.xml'. Reasons:
Rhino attempt: java is not defined
PhantomJs attempt: window is not defined
NodeJS attempt: EACCES, permission denied '/Users/TestReports'

i created the folder manually and still shows the above in the console

is 'Protractor/TestReports' the correct path and does that folder exist?

On Wed, Apr 15, 2015 at 5:01 PM, spatchamatla [email protected]
wrote:

@bloveridge https://github.com/bloveridge @spenoir
https://github.com/spenoir , i downgraded to [email protected] and
used the below

//For junit output reports
var reporters = require('jasmine-reporters');
var capsPromise = browser.getCapabilities();
capsPromise.then(function (caps) {
var browserName = caps.caps_.browserName.toUpperCase();
var browserVersion = caps.caps_.version;
var prePendStr = browserName + "-" + browserVersion + "-";
jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('Protractor/TestReports', true, true, prePendStr));
});

it started working, the only down side is its creating the folder in home
directory rather than in project folder. When i tried using the project
folder path, it still says

Rhino attempt: java is not defined
PhantomJs attempt: window is not defined
NodeJS attempt: Arguments to path.resolve must be strings

i created the folder manually and still shows the above in the console

—
Reply to this email directly or view it on GitHub
https://github.com/angular/protractor/issues/60#issuecomment-93460326.

'Protractor/TestReports' is the path that been created in the home folder, then i used

jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('../TestReports', true, true, prePendStr));

and i manually created

when i ran the test, it still gives
Warning: writing junit report failed for '../TestReports', 'CHROME-41.0.2272.118-TestImpersonationPage.xml'. Reasons:
Rhino attempt: java is not defined
PhantomJs attempt: window is not defined
NodeJS attempt: EACCES, permission denied '/Users/TestReports'

you could try relaxing the permissions on the TestReports folder, Not sure
after that.

On Wed, Apr 15, 2015 at 5:10 PM, spatchamatla [email protected]
wrote:

'Protractor/TestReports' is the path that been created in the home folder,
then i used

jasmine.getEnv().addReporter(new jasmine.JUnitXmlReporter('../TestReports', true, true, prePendStr));

and i manually created

[image: screen shot 2015-04-15 at 12 07 09 pm]
https://cloud.githubusercontent.com/assets/4671240/7163457/278cffa6-e368-11e4-9994-2c0b5d33f07c.png

when i ran the test, it still gives
Warning: writing junit report failed for '../TestReports',
'CHROME-41.0.2272.118-TestImpersonationPage.xml'. Reasons:
Rhino attempt: java is not defined
PhantomJs attempt: window is not defined
NodeJS attempt: EACCES, permission denied '/Users/TestReports'

—
Reply to this email directly or view it on GitHub
https://github.com/angular/protractor/issues/60#issuecomment-93466365.

Sure,Thanks @spenoir for your suggestions and input

great thread. helped a lot.
thought I had an issue - but don't. cannot find a way to delete :)
keep up the good work.

@jkmurthy @ksraja It seems that the jasmineReports listed is actually the export of the require function.

e.g.,
var jasmineReporters = require('jasmine-reporters');

The example on the jasmine-reporters readme works for me: https://github.com/larrymyers/jasmine-reporters#protractor

If you only need console output, I was pleased with https://github.com/bcaudan/jasmine-spec-reporter/blob/master/docs/protractor-configuration.md

@josephdpurcell the link you provided also worked for me. Here is my config, I hope it helps!
Make sure that you have installed the mkdirp package as well as the other jasmine 2.x and jasmine-reporters 2.X packages.
This config creates the folder to aviod a silent failure if the folder does not exists.

   var folderName = ('output_folder');
    var mkdirp = require('mkdirp');
    mkdirp(folderName, function(err) {
      if (err) {
        console.error(err);
       } else {
          var jasmineReporters = require('jasmine-reporters');
          jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter({
            consolidateAll: true,
            savePath: folderName,
             filePrefix: 'xmloutput-'
            }));
         }
    });