Electron: Headless version for testing

By using hidden browser window atom-shell can actually do what phantomjs does, the example in phantomjs's homepage could be translated to atom-shell:

BrowserWindow = require('browser-window');

console.log('Loading a web page');
var page = new BrowserWindow({show: false});
var url = 'http://www.phantomjs.org/';
page.on('loading-state-changed', (event, isLoading) {
  if (!isLoading)
    //Page is loaded!
    require('app').exit();
});
page.loadUrl(url);

Of course we may need to add some more APIs for automation testing purpose.

The only problem is instead of drawing into a virtual buffer, atom-shell actually draws the page into a real window, this would require a graphic environment, on Window and OS X it doesn't matter, however on Linux we have to use xvfb to provide a X server for atom-shell. This is by design in Chromium's content API, so we can not do anything to remove the dependency of graphic environment.

I've had success with this lately (using Xvfb on an Ubuntu server). My use-case is capturing screenshots of templated pages. In fact, I've found that atom-shell through Xvfb on the server (an m3-large) has better performance than on my local Macbook pro. This led me to want to get atom-shell working through Xvfb in osx as well.

Since osx ships with Xvfb, that part is easy. Can I get atom-shell to use an Xvfb display in osx? Using the standard DISPLAY env variable like I do in linux is not working. Perhaps libchromiumcontent doesn't know how to use X11 when running in Darwin?

Xvfb only works for applications that written for X11 environment, on OS X atom-shell uses Cocoa for display, which I believe can not work with Xvfb.

Yeah kinda put that together. Probably not really a way to compile from source for X11 instead?

@zcbenz it is currently not possible to create a BrowserWindow that is bigger than the current resolution on OS X, even if you use new BrowserWindow({show: false});

@FWeinb can you open a separate specific ticket for the above?

Created #475

I'm closing this because there is no way to draw a page without actually creating a native widget in Chromium, and I don't think they will ever allow it.

For automatic tests, we do support Selenium.

The CEF project has support for off-screen rendering, so you can draw the screen into a buffer instead of a window. Regarding X server for Linux, it seems there's a way to work without it by adding a target called Ozone (see discussion here).

@etiktin Thanks for the information! I'm reopening this since there is an existing implementation on how to do it.

We could really use support for this. We recently replaced Phantom with Electron in Nightmare and love it so far, but it doesn't work out of the box on Linux.

Here's what we need to do right now to get it working: https://github.com/segmentio/nightmare/issues/224#issuecomment-141575361

I've been given the task to automate user behavior for one of our web applications that will be converted to a stand alone desktop Electron application. Before our company decided to make this move, we created page objects using the chrome web driver and interacted with the web application by invoking buttons/dropdowns/textboxes using the css selectors. Is there a way to do this with a web application that is wrapped with the Electron shell? The company plans on using the menu bar options to invoke certain functionality and I tried to get to the default menu bar options like File/Edit/Help using the JavaScript driver with no success. Are there any examples out there on how to do this?

https://github.com/segmentio/nightmare/issues/224#issuecomment-141575361 it seems @matthewmueller's snippet works on Linux :+1:

Has anyone gotten headless testing working on SuSE? Specifically SLES?

@fritx The same used for SlimerJS, but that's NOT headless mode.

@fritx that's what @zcbenz was saying, you have to have Xvfb running. Both CEF3 and Chromium Content Shell currently depend on Xlib. But with completion of Ozone: https://www.chromium.org/developers/design-documents/ozone
you'd be able to provide any low level I/O.

Apparently, there is a master bug in Chromium itself: https://code.google.com/p/chromium/issues/detail?id=546953

This is interesting:

Date: Wed Dec 02 15:35:21 2015

[headless] Initial skeleton of headless/public/

Create outline of future Headless API.

Does ChromeDriver work with electron?

A headless binary that doesn't require xvfb would open up new environments such as AWS Lambda - sign me up!

@Vanuan Have you heard of Nightmare? That might help you if there's nothing you specifically need from ChromeDriver.

Does it have Capybara/Selenium driver?

+1

I'm a little confused. Is there a headless mode? Can we effectively do this with BrowserWindow({show: false})? This would be very useful to me, I'm trying to get this to work so that we can create server-side web components: https://github.com/scramjs/scram-markup-engine

I think I've answered my own question as I've been looking around. Electron does not natively support a sophisticated headless mode. Nightmare seems to allow something like it, but you still have to do some configuration to get it to work on certain systems with no graphical environment. Electron can also do it if you use BrowserWindow({show: false}), but you must use xvfb to provide a graphical environment on headless Linux systems (which doesn't seem too bad actually). Correct me if I'm wrong, and +1 to this feature.

With the new chromium headless project [1] would it be possible to make electron headless without using xvfb?

I believe the current limitation was with libchromium? Have the chrome guys fixed that?

1: https://chromium.googlesource.com/chromium/src/+/master/headless/README.md

There's a related forum: https://groups.google.com/a/chromium.org/forum/#!forum/headless-dev

Any progress on this? This would be really useful for testing

segmentio/nightmare is a perfect for this. Simply:

const nightmare = Nightmare({
  show: true
});

@amilajack For simple cases like running simple Mocha unit tests headlessly, Nightmare would be like using a 20-pound sledgehammer to drive in a small nail (read: massive overkill). It's a fully fledged, batteries included browser automation library that can not only perform basic navigation and input, but even save HTML and PDF files to disk or take screenshots. Exactly 0% of this library should be necessary to run simple unit tests.

@isiahmeadows @mcolyer said that he wanted a headless version of atom-shell that could be used as a replacement'. Electron is pretty much exactly that with extra features.

Yeah, but why would you need sugar for what you don't use? (I was referring to all the sugar - you could theoretically re-implement Electron in its entirety with just vanilla Node + OpenGL bindings).

The most common use case for headless browsers are things like what mocha-phantomjs and Karma already exist for - running browser unit tests from the CLI. Most people use xvfb, a headless X server, on Travis if they need to test Firefox/Chrome, because that doesn't have a running X server, and you could even run Electron with that, but headless browsers like PhantomJS and SlimerJS don't need an X server. Electron + Nightmare still needs an X server of some kind (even if it's xvfb) to run, and this issue is asking for that dependency to be removed, but it most likely won't happen until Chromium itself can go headless and those changes propagated to libchromiumcontent.

Headless is now in Chrome 59: https://www.chromestatus.com/features/5678767817097216

@sindresorhus @zcbenz Will this change in Chromium make any difference here?

Electron is already wonderful, and a headless mode would make it even better!

(It would also be useful for Nightmare, which is based on Electron)

I was able to get Xvfb working on lambda which might be helpful for lambda-based testing...https://github.com/nisaacson/aws-lambda-xvfb

Any word on when Electron will support true headless? Can we count on this to happen? I can't wait to drop xvfb.

@lastmjs did you manage to get Electron to run on AWS Lambda based on xvfb?

@schickling See https://github.com/nisaacson/aws-lambda-xvfb

Thanks for your comment @MrSaints. I'm indeed debugging this repo for several hours now since I cannot manage to get nightmare to run. Does it work for you?

@zcbenz FYI chrome 59 will get headless mode support https://www.chromestatus.com/features/5678767817097216

@schickling see https://github.com/JohannesHoppe/nightmare-html2pdf -- nightmare in docker, with Xvfb

Thanks @JohannesHoppe, I got Nightmare working in Docker with Xvfb but want to run it on AWS Lambda instead.

I've opened an issue to replace Electron with headless Chrome in Nightmare: https://github.com/segmentio/nightmare/issues/1092

@schickling try this: https://github.com/dimkir/nightmare-lambda-tutorial

Apologies if this is answered elsewhere, but I can't find a concrete answer. In his highly +1'd comment above, @sandstrom pointed out that headless is now available in Chrome 59.

Is support for Chrome's headless flag on the development roadmap for Electron? It seems that this is a potentially huge "win" for Electron as it makes true headless usage possible.

I agree with @rinogo. Having a headless option for electron will be very usefull for running tests in ci systems and on dev box without needing virtual display and taking over the machine. I am also interested to know roadmap for electron and possibilly contribute.

Xvfb is annoying, It will be great if Electron support true headless!

xvfb-maybe in the meantime

There's a new article out from Google: https://developers.google.com/web/updates/2017/04/headless-chrome

It's coming soon.

Looks like Chrome 59 is on the stable channel now. What are the next steps to support headless in Electron?

Please, please make this a reality — by doing this and enabling NightmareJS to run headless Electron, you'd ultimately eliminate like a third of all Selenium use cases.*

* Being hyperbolic but also not really

@aendrew I reckon NightmareJS could be _rewritten_ to use / include a CDP backend instead, rather than kerfuffle over making Electron (with a philosophy: _build cross platform desktop apps_) headless. See: https://github.com/cyrus-and/chrome-remote-interface

@MrSaints I don't envy the maintainers; they _just_ finished converting it from PhantomJS like a year ago. Though if anything, perhaps it's good motivation to make Nightmare's browser layer pluggable...

Regardless, the point about Electron being desktop-oriented is well-taken.

@aendrew @MrSaints At the risk of broadcasting my ignorance... How difficult is this change (supporting the headless flag) to implement? I'm not sure how Electron interfaces with/extends Chromium, but it seems to me that the steps to implement this are:

1. Upgrade Electron's Chromium to version 59+.
2. Supply the headless command-line flag/configuration parameter.
3. Other stuff I'm obviously not accounting for. :)

I guess what I'm saying is, with headless now available in Chromium, implementing a headless mode (e.g. for NightmareJS) seems to be relatively straightforward. Granted, a general-purpose version of Electron might take some time to iron out certain use cases. However, to generate a build designed for something like NightmareJS, should be pretty straightforward, right?

If I had an immediate need for the speed improvements, I'd jump in and take a shot at it. However, we're still getting by with NightmareJS as-is, so we're ok for the time being.

Regardless, thanks to the maintainers of all of these projects for their contributions and hard work! :)

Regarding NightmareJS: We're currently working on a version of Nightmare built on headless Chrome even capable of running in serverless environments such as AWS Lambda. We'll open source it soon @graphcool. 🚀

@schickling that's gonna be awesome!

One thing to keep in mind with --headless in chrome is that it disables support for all plugins. So, if you need flash (which is doable with electron/nightmare) or the PDF viewer, etc, --headless is not for you and you'll want to use xvfb.

I can't wait to run Electron in AWS Lambda...I think

Amen @lastmjs Amen

What about this solution ?
https://github.com/dimkir/nightmare-lambda-tutorial

Haven't tried it yet though

@xplodedthemes uses a precompiled version of xvfb

Shameless plug: https://github.com/joelgriffith/navalia. This was built on top of headless Chrome for functional tests and more. Includes some nice features like parallelizing multiple tasks, a GraphQL front-end, and more.

I welcome any PRs/Issues/Feedback!

Hey everyone! Sorry to keep you all waiting... 💤

We've just open-sourced Chromeless. It's based on headless Chrome and works both locally and on AWS Lambda. The API is pretty similar to NightmareJS.

You can try it out here: https://chromeless.netlify.com

Has Chromeless just replaced Nightmare? Chromeless has a long way to go before it catches up to Nightmare.

We use it as a replacement for Nightmare as we need the ability to run multiple tests as the same time.

Question (maybe not good for this thread): are you considering making the pdf functionality work? If so, this could save us a TON of headache (and cost).

Wow, that's incredible. Good work guys!

@zcbenz it seems like solutions have emerged for this: is it ok to close?

Umm this issue is still extremely valid. The "solutions" all involve moving off of Nightmare completely.

Totally agreed with @keithkml - the solutions, although well-intentioned (thank you!), have been more along the lines of "alternatives" until an actual solution is provided. Along those lines, does someone here have sufficient expertise to answer the questions I posed in my previous comment? (And again, please excuse my ignorance! :))

I still don't get the point in responses. Could someone please give me a clarify that do we have NATIVE headless mode for electron app to run in the CI or not?

@hitliubo Chrome has a --headless flag, but it only works if you're not using plugins like Flash or the PDF reader. If you are, then the answer is an affirmative no at the moment. If you are not, you could pass that flag (along with --disable-gpu if necessary - they fixed this missing implication in newer Chrome versions IIRC) when creating the browser context and see if that works. (Note that if you're using something like Nightmare and you fall into the second category, you really should file an issue in that project's respective repo if there isn't one already.)

Even if you can't use --headless (or if it doesn't work), there are options:

• Windows: Unless you're using something obscure, you'll always have a window context to render into. Windows Server 2016 dropped default GUI support, but it still supports creating GUI windows, and AFAICT should support the bare minimum for Selenium testing.
• macOS: You'll always have a GUI here (and thus have a window context to render into). Apple doesn't provide a version that doesn't.
• Linux: xvfb is a headless X server that's available for just about every common Linux distro (and I mean "common" pretty loosely here). If you're using Travis, they have instructions on how to add it to your project.

@isiahmeadows Thanks for your information. Currently I have a web app running in browser and with chrome/firefox I could always add --headless for the headless testing. Recently we'd like to convert it to Electron app and I tried with --headless which does not work in my macOS. Now I know thee reason. Thanks!

Actually I don't like the solution by xvfb as it's not native. However, given that we don't support native headless so I guess I need to give it a try.

FYI - Now I'm using capybara for the testing.

That would be great (y)

Is there any update on this?

I got redirected here from a post about rendering directly to a linux frame buffer, but this seems to be focused on headless testing. Has any progress been made rendering directly to a _real_ frame buffer?

@quinn I'm pretty certain you're going to need an X server, although you can run X11 (Xorg) on a frame-buffer if you like (see: https://www.x.org/releases/current/doc/man/man4/fbdev.4.xhtml).

_Edit:_

Actually after looking into this a little bit more, this can also be achieved by using ozone. ( https://github.com/jakwu/ozone-fb )

Adding ozone would also allow wayland support, another feature that electron is missing seeing majority of Linux distributions have since migrated to this.

Based on the description of ozone-fb and @isiahmeadows comment, it seems GPU acceleration does not work if rendering to a frame buffer.

@trusktr That's a 2 year old comment. I'd recommend not taking my comment as authoritative, as that could have changed (I haven't checked since then).

I add headless lib to electron and call the HeadlessShellMain。
run:

e run  --headless --enable-logging --v=2 --disable-gpu --screenshot  http://192.168.50.206

then it shows:

Running "/home/a/dev0/e9.2.1/src/out/ReleaseSym0/electron --headless --enable-logging --v=2 --disable-gpu --screenshot http://192.168.50.206:8889"
[1028/172650.483932:INFO:cpu_info.cc(53)] Available number of cores: 4
[1028/172650.484061:VERBOSE1:zygote_main_linux.cc(217)] ZygoteMain: initializing 0 fork delegates
[1028/172650.484400:INFO:cpu_info.cc(53)] Available number of cores: 4
[1028/172650.484465:VERBOSE1:zygote_main_linux.cc(217)] ZygoteMain: initializing 0 fork delegates
[1028/172650.493514:VERBOSE1:webrtc_internals.cc(119)] Could not get the download directory.
[1028/172650.494623:VERBOSE1:proxy_config_service_linux.cc(500)] All gsettings tests OK. Will get proxy config from gsettings.
[1028/172650.494764:VERBOSE1:proxy_config_service_linux.cc(1261)] Obtained proxy settings from annotation hash code 11258689
[1028/172650.494873:VERBOSE1:proxy_config_service_linux.cc(500)] All gsettings tests OK. Will get proxy config from gsettings.
[1028/172650.494919:VERBOSE1:proxy_config_service_linux.cc(1261)] Obtained proxy settings from annotation hash code 11258689
[1028/172650.504033:VERBOSE1:sandbox_linux.cc(69)] Activated seccomp-bpf sandbox for process type: renderer.
[1028/172650.505596:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.511468:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.524408:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.524916:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.525173:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.525963:VERBOSE1:sandbox_linux.cc(69)] Activated seccomp-bpf sandbox for process type: gpu-process.
[1028/172650.526373:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.528735:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.531839:VERBOSE2:thread_state.cc(470)] [state:0x556bd75583a0] ScheduleGCIfNeeded
[1028/172650.535051:ERROR:paint_controller.cc(646)] PaintController::FinishCycle() completed
[1028/172650.550076:VERBOSE1:configured_proxy_resolution_service.cc(873)] PAC support disabled because there is no system implementation
[1028/172650.550312:VERBOSE1:configured_proxy_resolution_service.cc(873)] PAC support disabled because there is no system implementation
[1028/172650.550923:VERBOSE1:network_delegate.cc(32)] NetworkDelegate::NotifyBeforeURLRequest: http://192.168.50.206:8889/
[1028/172650.575829:VERBOSE1:url_loader.cc(418)] To buffer: http://192.168.50.206:8889/
[1028/172650.580122:VERBOSE1:v8_context_snapshot.cc(153)] A context is created from snapshot for main world
[1028/172650.587399:VERBOSE1:v8_context_snapshot.cc(153)] A context is created from snapshot for main world
[1028/172650.595294:ERROR:paint_controller.cc(646)] PaintController::FinishCycle() completed
[1028/172650.612295:ERROR:paint_controller.cc(646)] PaintController::FinishCycle() completed
[1028/172650.676553:INFO:headless_shell.cc(620)] Written to file screenshot.png.

Does that mean headless has been implemented?

@bigben0123 thats interesting and very exciting! So you have compiled your own version of electron incorporating the headless shell from chromium?

Have you tested in an environment without X on Linux to see if it works?

When chromium is run in headless mode the render sub-processes are started with the passed through —headless flag (you can see this is you use ‘ps args’ from memory). Is this happening for you?

(Strangely with electron currently if you try to start it with —headless the flag isn’t passed though to the render processes but is to the GPU process.)

@bigben0123 thats interesting and very exciting! So you have compiled your own version of electron incorporating the headless shell from chromium?

Have you tested in an environment without X on Linux to see if it works?

When chromium is run in headless mode the render sub-processes are started with the passed through —headless flag (you can see this is you use ‘ps args’ from memory). Is this happening for you?

(Strangely with electron currently if you try to start it with —headless the flag isn’t passed though to the render processes but is to the GPU process.)

Yes, I run it on ubuntu which just start in user command mode.
Headless has been passed:

electron --headless --enable-logging --v=2 --disable-gpu -print-to-pdf http://www.google.com
electron --type=zygote --no-zygote-sandbox --enable-logging --headless --v=2 --headless
electron --type=zygote --enable-logging --headless --v=2 --headless
electron --type=zygote --enable-logging --headless --v=2 --headless
electron --type=gpu-process --field-trial-handle=15536072708541054845,15522400966085077738,131072 --enable-logging --headless --v=2 --headless --gpu-preferences=MAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAABgAAAAAAAQAAAAAAAAAAAAAAAAAAAACAAAAAAAAAA= --use-gl=swiftshader-webgl --override-use-software-gl-for-tests --enable-logging --v=2 --shared-files
electron --type=utility --field-trial-handle=15536072708541054845,15522400966085077738,131072 --lang=en-US --service-sandbox-type=network --enable-logging --use-gl=swiftshader-webgl --v=2 --headless --enable-logging --v=2 --shared-files=v8_snapshot_data:100
electron --type=renderer --allow-pre-commit-input --enable-logging --v=2 --field-trial-handle=15536072708541054845,15522400966085077738,131072 --disable-databases --disable-gpu-compositing --lang=en-US --headless --lang=en-US --num-raster-threads=2 --enable-main-frame-before-activation --renderer-client-id=4 --shared-files=v8_snapshot_data:100
electron --type=broker

@bigben0123 do you have your changes somewhere? Even if this doesn’t make it into electron core for some reason I would love to be able to use it.

This commit only merge chrome headless lib to electron and you can use it the same way as chrome does.
https://github.com/bigben0123/electron/commit/b6cad8993d68d39f1732aa6ed5ece0135b9ae0c8

As far as I'm concerned, chrome and headless has different content layer implementation. Like two browser shell, if you start headless it's nothing to do with chrome, only except you start by "chrome --headless".

The one of headless goal is that ”Minimize the number of invasive or headless-specific changes (e.g., #ifdefs) to Chromium’s code base“.

Therefore, it's difficult to implement that electron is headless to remove xvfb. We can just let electron support headless, but you cannot execute electron apps.

We may use headless' implementation to replace electron's to get a new headless branch.

remove the dependence from electron/BUILD.gn:

      "//ui/events/devices/x11",
      "//ui/events/platform/x11",
      "//ui/gtk"  #all of gkt related

    if (use_x11) {
      deps += [
        "//ui/gfx/x",
        "//ui/gtk:x",
      ]
    }
    configs += [ ":gio_unix" ]
    configs += [ "//build/config/linux:x11" ]

replace with:
"//ui/display",
"//ui/events/devices",