Jsdom: Support for WebComponents API

It'd be interesting to look into which APIs webcomponents.js uses that jsdom doesn't support. If I had to guess, that will be much easier to implement than the full web components spec.

That said, it would be pretty cool to implement web components. Probably not as hard as one might think---the specs are relatively small.

Just had time to dig into this a bit:

First off, we don't have Window defined in the window scope. I just patched this with this.Window = this.prototype in the Window constructor.
Second, webcomponentsjs expects Window to have another prototype, i.e. the EventTarget prototype, which we don't implement as a seperate entity.

Just a bit of info, because I had a bit of time.

Nice. Should be able to expose Window pretty easily. EventTarget prototype is a bit trickier but seems doable given how we currently implement that stuff; it's been a TODO of mine.

Okay, patches so far are rather easy:

• [x] this.Window = Window; in the Window constructor
• [x] inherits(dom.EventTarget, Window, dom.EventTarget.prototype); after the definition of Window

The next crash of webcomponents.js happens due to us not implementing HTMLUnknownElement (#1068), after shiming that we need to implement the SVGUseElement. That's what I'm currently blocked on, because webcomponents.js apparently doesn't like the SVGUseElement shimmed by a HTMLDivElement and throws in an assert.

Okay I checked into the Polyfill some more, we need to implement/you need to shim the following:

• [x] HTMLUnknownElement #1068
• [ ] SVGUseElement
• [ ] window.CanvasRenderingContext2D
• [ ] Range APIs (including: document.getRange(), window.getSelection(), window.Range, window.Selection; #804 might be a start)
• [ ] npm i canvas

(non-exhaustive list for now)

A start is something like the following:

jsdom.env({
  file: __dirname + '/index.htm', // refers to webcomponent.js
  created: function (err, window) {
    jsdom.getVirtualConsole(window).sendTo(console)

    window.document.createRange = function () { }
    window.getSelection = function () { }
    window.Range = function () { }
    window.Selection = function () { }
    window.CanvasRenderingContext2D = function () { } // Object.getPrototypeOf(require("canvas")(0,0).getContext("2d")) might be better
    window.SVGUseElement = window.HTMLUnknownElement
  },
  done: function (err, window) {
    console.log(err[0].data.error);
    console.log(window.CustomElements)
  },
  features: {
    ProcessExternalResources: ['script']
  }
});

That done, there's some bug in our HTMLDocument constructor, which leads to a maximum call stack error. The constructor is at the moment only for internal use, however it's valid that some script on the site makes calls to it so we need to make that constructor available for public consumption.

+1 Would love to see WebComponents on jsdom, particularly as Polymer gains in popularity, would be great to be able to test custom elements on a headless system.

Right now there is no cross-browser definition of web components, so it'd be premature to implement. (We're not just going to copy Chrome.) In the meantime, you can try using Polymer with jsdom.

@domenic fair enough. Well it's more the support for the WebComponents.js polyfill that I'm after, as that's what Polymer depends on - or webcomponents-lite (polyfills all of them barring Shadow DOM) at the moment. Made a few attempts to get Polymer working on jsdom, but no luck so far - I'm assuming @Sebmaster's tasks in the comment above will at least need to be patched first.

My understanding is that there are three separate polyfills in question. The one in the OP is separate from Polymer. Then there's the webcomponents.org polyfills, which used to be used in old-Polymer. Then in Polymer 1.0, they have their own polyfills, I think, which aren't really polyfills, but instead alternate libraries that do things kinda web-component-ish. Maybe that is webcomponents-lite though.

On the WebComponentsJS repo, it says that the webcomponentsjs-lite is a variant, providing polyfills for all _but_ Shadow DOM, which Polymer then independently attempts to shim using their Shady DOM system. So from that I'm pretty sure Polymer relies on WebComponents as much as it can, with the WebComponentsJS polyfill doing the grunt work. The lite version is supposed to be significantly less weight (funnily enough..) so I'll see if I can pinpoint what it is that jsdom needs for the lite version. What do you think the chances are of getting the polyfill (lite or full) working in jsdom is?

It's really hard to say without some investigation... looking forward to what you find out.

Yeah, I think my list of todo tasks is still applicable and required to use the shims. Getting #1227 merged in might make us a lot quicker with implementing standards-compliant interfaces so we can fix the missing ones more quickly.

I've (probably naively) started working on adding CustomElementsRegistry as a way to understand how jsdom is structured. I added "custom-elements/custom-elements-registry/define.html" to the web platform tests list and it passes when it shouldn't (i haven't implemented nearly enough yet). I'm pretty sure the test isn't really running as even adding a throw at the top of the test won't prevent it from passing. So I've obviously missed something; aside from adding the test in test/web-platform-tests/index.js is there anything else I need to do?

Seems like that's caused because we fail in the initial const testWindow = iframe.contentDocument.defaultView; line because contentDocument is undefined for some reason. Might be an issue with our loading order vs. script execution, but haven't dug into that. Hope that helps you work around that. We might have to simplify the test for our purposes (for now).

That helps very much, thanks! I'll see if I can figure out what is going on there, and if not I'll create a simplified test as you recommended.

@Sebmaster Just in case your interested, I did a bit of research into what is going on with that test and the results are surprising to me.

The test is using the named access feature of html. This means you can do stuff like:

<div id="foo"></div>
<script>
  console.log(window.foo === document.getElementById('foo'));
</script>

_However_, if the element has a nested browsing context, the global should point to that instead (see the linked spec). For iframe's that's the contentWindow. jsdom gets this right, there's even a test. Safari gets it right too.

What's crazy is that Chrome and Firefox get this wrong; the global points to the iframe, not it's contentWindow. Seeing this, I assumed it was a jsdom bug and did some hunting, eventually finding that test, which led me to the spec.

tldr; working on jsdom is very educational and you guys do an amazing job.

Going to file bugs in the respective browsers. Also will send a PR to web-platform-tests, I found some other mistakes in the test as well.

This is even more motivation to upstream tests like https://github.com/tmpvar/jsdom/blob/master/test/living-html/named-properties-window.js to WPT. Thank you for posting! It makes me feel really great about jsdom ^_^

Hi!

I managed to make Custom Elements polyfill work with jsdom by combining

• this repo: https://github.com/RisingStack/nx-seo
• with this custom element polyfill: https://github.com/WebReflection/document-register-element

Note: the repo uses jsdom 8.5.0. The reason is that I only had success with a MutationObserver polyfill, that uses Mutation Events internally. Mutation Events were removed after 8.5.0 due to bad performance. If native Mutation Observer comes out I will remove the polyfill and update to the latest jsdom.

I've got the latest jsdom, and https://github.com/WebReflection/document-register-element is working for me! I've been experimenting with the more official polyfills, and I'm having trouble for some reason. My goal is to get at least custom elements and html imports to work...it would be awesome if we could get Polymer to work as well.

I can get the Polymer scripts to run without error. I can even create a component and pass it to the Polymer constructor. After that it fails silently. I think shadow DOM is the issue.

I've been trying to get the webcomponentsjs HTML imports polyfill to work. I can get the script to run, and I believe my HTML imports execute an xmlhttprequest, but it doesn't seem like the scripts in my imports get run.

Care to share an example @lastmjs? I'm currently knee deep in web components myself. If I can be of help i'd gladly contribute with you.

@snuggs Thanks! Give me a day or two, I'm in the middle of some pressing things at the moment.

@snuggs If we can get the webcomponents-lite polyfill to work, we should be able to use Polymer. Shadow DOM seems like the hardest polyfill to get working so far, and if we use webcomponents-lite we won't have to worry about that for the time being, because we'll have access to template, custom elements, and HTML imports.

I can get HTML imports to work with the webcomponents-lite polyfill. I was running into some weird behavior, then I came across this: https://github.com/Polymer/polymer/issues/1535 It looks like HTML imports can only be loaded over a cors-enabled non-file protocol. So I spun up a quick http server in the root directory of my project:

npm install -g http-server
http-server --cors

And here is the basic code I've been working with:

const jsdom = require('jsdom');

const doc = jsdom.jsdom(`
    <!DOCTYPE html>

    <html>
        <head>
            <script src="bower_components/webcomponentsjs/webcomponents-lite.js"></script>
            <link rel="import" href="http://localhost:8080/bower_components/polymer/polymer.html">
        </head>

        <body>
            <test-app></test-app>

            <dom-module id="test-app">
                <template>
                </template>

                <script>
                    setTimeout(() => {
                        class TestApp {
                            beforeRegister() {
                                this.is = 'test-app';
                                console.log('before register');
                            }

                            ready() {
                                console.log('ready');
                            }

                            created() {
                                console.log('created');
                            }

                            attached() {
                                console.log('attached');
                            }
                        }

                        Polymer(TestApp);
                    }, 1000);
                </script>
            </dom-module>
        </body>
    </html>
`, {
    virtualConsole: jsdom.createVirtualConsole().sendTo(console)
});

For some reason I have to wrap the TestApp instantiation in a setTimeout. It seems like the Polymer HTML import is not blocking the rendering of the rest of the HTML, so without the setTimeout the Polymer constructor is not defined. Is that normal behavior for HTML imports?

beforeRegister does get called, so the Polymer constructor is doing something. So now we've effectively got HTML imports, of course templates working with the webcomponents-lite polyfill. I'm not sure how the custom elements polyfill is faring.

When I put inside of the TestApp class a ready or created method, they are not called. Seems like the lifecycle events aren't being handled properly. The root of that problem might be in the implementation of the custom elements polyfill. I'll keep playing around.

Potential problems to be solved:

• [ ] HTML imports not blocking properly
• [ ] custom element polyfill working or not?
• [ ] Polymer lifecycle methods not being called

More tinkering is leading to more insights. I think the order of imports and registrations might be messing things up for us. When I run const testApp = document.createElement('test-app'); after the Polymer constructor is called, the created and ready methods are called, but not the attached method. Perhaps jsdom is not handling custom element literals correctly? Also, even when calling document.body.appendChild(testApp), the attached lifecycle method is never called.

This might help with understanding the load order: https://github.com/webcomponents/webcomponentsjs#helper-utilities

@lastmjs i've been currently flipping coins between CustomElementRegistry.define() and document.registerElement(). I saw Domenic has given some great input and merged some work relative to whatwg (https://github.com/whatwg/html/issues/1329) It seems like there's an API migration going on. For instance I believe the spec calls connectedCallback which is paired with attachedCallback functionality. Also assuming you meant attachedCallback when you said attached as that handler isn't part of the API. I've experienced define() and registerElement() firing different callbacks respective to each method. I've figured out the custom elements strategy. HTMLImports Domenic mentioned before an implementation that uses an XMLHTTPRequest patch. I believe can convert directly to a DocumentFragment directly from the response. Smells like that could be snakeoil with the "imports". A "faux" import may be where sanity lives.

There also seems to be some fockery with super() being called on HTMLElement when transpiling from ES6 -> ES5 so keep a heads up for that. I've been experiencing this with Rollup.js/Babel and was forced to use the (lightweight) shim from webcomponents package.
https://developers.google.com/web/fundamentals/getting-started/primers/customelements

Lastly seems I get (more) success when I create with a prototype tag.

document.createElement('main', 'test-app')

As @domenic mentioned to me before we want to be cautious to implement lowest common denominator specs and not just do what GOOGLE does. Seems like the lines are blurred with web components. But I'm a fan.

What methods have you been working with?

So far I've mostly been playing with the webcomponents-lite polyfills only, and Polymer < 2.0. So when I mentioned the attached method I meant the Polymer lifecycle method that they use instead of the attachedCallback. Also, as far as I'm aware the polyfills haven't switched over to the new v1 custom elements spec yet. So everything I'm playing with is only in the hopes of getting Polymer to work with the current polyfills.

@snuggs Are you using polyfills right now or are you working on an actual implementation in jsdom?

@lastmjs I don't use polyfills as I feel it's not necessary to get 80% of the way. The platform is mature enough now that with a little upfront tweaking can just use the native constructs. I like to use lightweight (usually hand rolled) tools instead of frameworks. That said that's not most people. Seems the intent Domenic has is Custom Elements 👍 html imports 👎 but no issue extending XMLHTTPRequest to handle the fetching of the document which would get us there. That was about 6 months ago. Much has changed since in implementation. Quite possibly thinking. So where do we finish @lastmjs?

@snuggs Perhaps the most sane and future-proof thing to do is to implement first-class support for Custom Elements and Shadow DOM in jsdom. Both standards are at v1 and it seems likely from what I'm hearing that the majority of the major browsers will be implementing them. How should we go about this? I have limited time right now, but perhaps we can lay out what needs to be done at least. @domenic Do you have any suggestions on how to move forward with these implementations, or any reasons why we shouldn't?

No concrete suggestions from me, besides just implementing the spec :)

I have a branch where I worked on this some time ago (spec has changed a bit since then). Implementing CustomElementsRegistry was easy enough, where I struggled was figuring out how to weave in custom element reactions into the codebase and when those should be called and from where. If I were to pick this back up (no promises) that's probably what I would focus on.

@matthewp That sounds helpful, where can I find that branch?

@matthewp yeah that would be nice

https://github.com/matthewp/jsdom/commits/custom-elements like I said, the spec changed since then, so it's out of date. And this is the easiest part, but it's a starting point if anyone wants it. @snuggs @lastmjs

• Templates are adopted by all major browser (W3C spec)
• Custom elements will be soon (W3C spec)
• HTML import will not be implemented by browser vendor anytime soon (W3C spec)
• Shadow dom is in between...(W3c spec)

(http://jonrimmer.github.io/are-we-componentized-yet/)

Personally simply supporting Custom element would be already great.

(Note that my understanding is that phantomJS 2.5 should be supporting at least Templates and maybe Custom element as they are moving on the more recent version of Webkit, not sure which one).

Actually, I mock the customElements, using the lib document-register-element

const {before} = require('mocha')

before(mockDOM)
before(mockCustomElements)

function mockDOM() {
  const {JSDOM: Dom} = require('jsdom')
  const dom = new Dom('<!doctype html><html><body></body></html>')
  global.document = dom.window.document
  global.window = document.defaultView
  window.Object = Object
  window.Math = Math
}

function mockCustomElements() {
  require('document-register-element/pony')(window)
}

Awesome, have you had any issues?

until now, no :D

but I need write more specs, cover more things to feel better

Awesome to see there is a way. As much as I like polymer, the test setu is hell and having jsdom as a fallback is nice ;) Thanks for putting the work in

Looks like there's a PR moving this forward! https://github.com/tmpvar/jsdom/pull/1872

Actually, I mock the customElements, using the lib document-register-element @darlanmendonca

Should read this link about attaching jsdom globals to node global. It's an anti-pattern.

Hello all,
I'm a bit confused regarding the status of running Polymer inside JSDOM (using Node.js 6.7.0 and JSDOM 11.1.0). I've tried various things, with mixed results. I'd be really grateful if somebody could fill me in here...

What I did so far:

1) I fired up a http server from my root directory

./node_modules/http-server/bin/http-server --cors

2) I loaded one of my Polymer components into JSDOM:

jsdom.JSDOM.fromURL("http://localhost:8080/path/to/my-component.html",
  { runScripts: "dangerously",
    resources: "usable"
  })
.then(function (dom) {
  setTimeout(() => {
    window = dom.window;
    component = window.document.querySelector("my-component");
  }, 10000);
})

(I also tried loading the component file from the file system, with the same results.)

3) This is my component code:

<!DOCTYPE html>
<html>
<head>
  <script src="/bower_components/webcomponentsjs/webcomponents-lite.js"></script>
</head>

<body>
<link rel="import" href="/bower_components/polymer/polymer.html">
<dom-module id="order-app">
  <template>
    <h1>Hello Polymer</h1>
  </template>

  <script>
    console.log("javascript is being executed");
    addEventListener('WebComponentsReady', function () {
      console.log("web components are ready");
      Polymer({
        is: 'order-app'
      });
    });
  </script>
</dom-module>
</body>
</html>

(I added the HTML head in order to load the webcomponents polyfill.)

What can I observe?

When I run this, I see

• that the webcomponents polyfill is being loaded from the webserver
• the message "javascript is being executed" in the console

What I don't see

• that the polymer.html component is being loaded from the webserver
• the message "web components are ready" in the console

This leads me to the conclusion that the WebComponentsReady event is not being fired (probably because the HTML import does not work?). Also,
window.WebComponents contains { flags: { log: {} } } -- the ready indicator is missing.

I also tried some mocking and polyfilling:

  window.Object = Object;
  window.Math = Math;
  require('document-register-element/pony')(window);

but that didn't seem to change anything.

Now, I'm wondering :-) Is this supposed to work at all? If so, why does it not work for me? If not, what is missing / required to get it to work?

Thanks for any insights!

Moin

I even tried that with even less success and I gave up to wait what will be the result of this discussion here.

https://github.com/sebs/noframework/blob/master/test/configurator.js

not a solution just another failed attempt. Same confusion btw. Same conclusion as well

Polyfilling custom elements in jsdom is proven to be very challenging. Can someone list the challenges of getting that implemented in jsdom? Trying to assess the level of effort to get that in.

The fundamental obstacle is that jsdom shares constructors and their prototypes.

This makes it basically impossible to implement a per-window custom elements registry, because the HTMLElement constructor is shared between all windows. So when you do the super() call in your custom element constructor, the now-running HTMLElement constructor doesn't know what window to look things up in. This sucks.

I'm not sure if there are any good intermediate solutions. The big gun is to move jsdom to an architecture that allows non-shared constructors/prototypes. We could do this in a few ways, all with different tradeoffs. Perhaps we'd want to open a dedicated issue to discussing it with the team and community, but for now let me list the ones that come to mind off the top of my head:

• Use [WebIDL2JSFactory] for everything in jsdom, or at least HTMLElement and all its descendants. I'm not sure if [WebIDL2JSFactory] even works well with inheritance yet, but it could be made to work. This alternative causes everyone to pay the cost of extra constructors/prototypes, but maybe that is better than making custom elements an opt-in feature.
• Have an option where jsdom runs all of the class definition modules inside of the vm sandbox. E.g. have some build step that bundles up all the web APIs in jsdom, then when you create a new window, we do vm.runScript() with that bundle inside the new sandbox global. This would probably allow us to get rid of [WebIDL2JSFactory].

I guess another solution would be to implement custom elements with a giant warning that the custom element registry is global per Node.js process? That seems terrible though.

After that initial hurdle, the rest is relatively straightforward in terms of following the spec. The hardest part will probably be implementing [CEReactions] and updating all our IDL files to have that in the appropriate places, but it's not too difficult.

I have been thinking about having a separate prototype version as well. Here are some of my thoughts.

I'm not sure if [WebIDL2JSFactory] even works well with inheritance yet, but it could be made to work.

No it does not, and I'm not sure how exactly to make it work. The second solution is much more straight-forward in my opinion.

Have an option where jsdom runs all of the class definition modules inside of the vm sandbox.

This is what I would prefer. The main problem is passing impl classes into the vm sandbox during initialization, though that can be done by putting everything from the outside context into one global property, and delete that global property after it is done. It would also allow properly implementing [NamedConstructor] and a couple of other extended attributes, and maybe even generating a V8 startup snapshot for a jsdom environment if someone is daring enough.

The whole [WebIDL2JSFactory] business was a hack in the first place, and I'd love to get rid of it as soon as possible.

+1 comments are not helpful to developing this feature, so I'm deleting at least one recent one.

Hi, I didn't realize @TimothyGu was working on this.
I actually have custom element registration & creation working at
https://github.com/mraerino/jsdom/tree/custom-elements-spec

I'm trying to be as minimally invasive as possible and as well to stay as close to the spec as possible.
Custom Element Registry web platform tests are passing.

While hacking on this last night I found a solution that works without modifying webIdl2JS.
See here: https://github.com/mraerino/jsdom/commit/592ad1236e9ca8f63f789d48e1887003305bc618

@TimothyGu would you be willing to combine forces on this one?

Just some updates here:
I am pretty confident about my implementation of the spec, but am currently stuck because of the [HTMLConstructor] extended IDL attribute. That's why I opened https://github.com/jsdom/webidl2js/issues/87

In the meantime, I will implement the [HTMLConstructor] algorithm using a [Constructor] attribute to be able to easily switch later. (I initially implemented it by inserting a mock HTMLElement class into window, but this didn't seem right.)

Yeah, as noted in https://github.com/tmpvar/jsdom/issues/1030#issuecomment-333994158, implementing HTMLConstructor correctly will require fundamental changes to jsdom's architecture.

Do you have any information on how many of the web platform tests your version is passing?

Just the customElementRegistry ones for now, and I could be totally wrong about my progress.

Edit: Ok, after re-reading your comment, I got what you mean. I'll try it with my implementation, but @TimothyGu also seems to be working on the separation.

I use Polymer so I am :+1: on this request feature

@dman777 @mraerino Same for slim.js developers. Slim uses native web components API and cannot inherit HTMLElement without hacks on jsdom.

Three years have passed since this issue was opened. Can anyone say when approximately jsdom will support custom elements?

TL;DR

I worked over the last 2 weeks on evaluating the feasibility to add support for custom element in jsdom. Here is the result of the investigation.

You can find a spec compliant implementation of custom element here: https://github.com/jsdom/jsdom/compare/master...pmdartus:custom-elements?expand=1. There are still some rough edges here and there, but at least most of the WPT-test pass. The remaining failing tests are either known JSDOM issues or minor issues that can be tackled when we will tackle the actual implementation in jsdom.

Now here is the good news, now that Shadow DOM is supported natively, with both the custom-element branch and mutation observer, I was able to load and render the latest version of the Polymer 3 hello world application example in jsdom 🎉. In its current state, the branch is not able to load a Stencil application (Stencil dev mode requires some unsupported features like module, and the prod mode throws for an unknown reason).

Action plan

Here is a list of the changes that need to happen first before, starting to tackle the actual custom element spec implementation. Each item in the list is independent and can be tackled in parallel.

Support for [CEReactions] IDL extended attributes

One of the core functionally missing in jsdom for adding support for custom elements is the [CEReactions] attributes. I was partially able to get around this issue by patching the right prototype properties. This approach works as long as the custom element reaction stack is global and not per unit of related similar-origin browsing contexts since all the interfaces prototypes are shared.

This approach has some shortcomings since some interfaces have the [CEReactions] attributes associated with indexed properties (HTMLOptionsCollection, DOMStringMap). Internally, jsdom uses Proxies to track mutation to those properties. The prototype patching of the interface doesn't work in this case. Another approach, to get around this issue would be to patch the implementation instead of the interface (not implemented).

I am not familiar enough with the webidl2js internal, but we should explore adding a global hook for [CEReactions]?

Changes:

• ce-reaction.js:
  
  
  
  • patching
  
  
  • CEReactions implementation
  
  

Support for [HTMLConstructor] IDL extended attributes

As @domenic explained above, adding support for [HTMLConstructor] is one of the main blockers here.

I was able to get around this issue here by patching the interface constructor for each browsing context. The interface constructor would be able to access the right window and document object while keeping the shared prototype. This approach also avoids the performance overhead of re-evaluating the interface prototype for each new browsing context.

I am not sure if it's the best approach here, but it fits the requirements without introducing extra performance overhead.

Changes:

• create-element.js

Make the make fragment parsing algorithm spec compliant (#2522)

As discussed here, the HTML fragment parsing algo implementation used in Element.innerHTML and Element.outerHTML is incorrect. The parsing algo need to get refactored, for the custom element reactions callbacks to get invoked properly.

Changes:

• parser changes

Improve interface lookup for create element algo

One of the issues that I quickly stumbled upon, was the introduction of new circular dependencies when adding support for custom element creation. Both CustomElementRegistry and create element algorithm requires access to the Element interfaces, creating a circular dependencies nightmare.

The approach taken in the branch was to create an InterfaceCache, that would allow interface lookup by element namespace and name but also by interface name. The interface modules are lazily evaluated and cached once evaluated. This approach gets rid of the circular dependencies because the interfaces are not required at the top level anymore.

This is one approach to resolve this long-standing issue in jsdom, one of this issue with this approach is that it would maybe break webpacked / browserified version of jsdom (not tested).

Changes:

• create-element.js

~Fix Element.isConnected to support Shadow DOM (https://github.com/jsdom/jsdom/pull/2424)~

This is an issue that slipped in with the introduction of the shadow DOM the isConnected returns false if the element is part of a shadow tree. A new WPT-test needs to be added here, since no test is checking this behavior.

Changes:

• Node-impl.js

Fix HTMLTemplateElement.templateContents node document's (#2426)

The template contents as defined in the spec has a different node document than the HTMLTemplateElement itself. jsdom doesn't implement this behavior today and the HTMLTemplateElement and it template contents share the same
document node.

Changes:

• HTMLTemplateElement-impl.js
• htmltodom.js. This change also has some downstream effect on the parser. If the context element is an HTMLTemplateElement, the HTML fragment parsing algorithm should pick-up the document node from the template content and not from the element itself.

Add missing adoption steps to the HTMLTemplateElement (#2426)

The HTMLTemplateElement needs to run some specific steps when it's getting adopted into another document. As far as I know, it the soil interface to have a special adoption step. The adopt node algorithm implementation would also need to be updated to invoke the this adopt step.

Changes:

• HTMLTemplateElement-impl.js
• Document-impl.js

Add support for isValue lookup in parse5 serializer

The serializing HTML fragments algo, when serializing an Element look up the is value associated with the element and reflects it as an attribute in the serialized content. It would be interesting to add another hook in the parse5 tree adapter, that would lookup the is value associated with an element getIsValue(element: Element): void | string.

An alternative approach (not implemented) would be to add change the behavior of the current getAttrList hook to return the is value to the attribute list, if the element has an associated is value.

Performance

Before doing any perfomance optimization I also wanted to check the performance of the changes in the branch. The addition of custom elements adds a 10% performance overhead compared to the current result on master for tree mutation benchmarks. However the creation of new JSDOM environment is now 3x to 6x slower compared to master, it would require deeper investigation to identify the root cause.

More details: here

@pmdartus this is very promising, excellent work! I've been using my hack branch jsdom-wc, for lack of a better option. I'm seeing some odd behavior and was hoping to swap out for your branch, but I'm hitting issues.

I register custom elements like:

class Component extends HTMLElement {

}

customElements.define('custom-component', Component);

But if I do:

const el = assign(this.fixture, {
  innerHTML: `
    <custom-component></custom-component>
  `,
});

I get an immediate: Error: Uncaught [TypeError: Illegal constructor].

Any thoughts on this?

The following snippet of code runs properly on the custom-elements branch on my fork: https://github.com/pmdartus/jsdom/tree/custom-elements

const { JSDOM } = require("jsdom");

const dom = new JSDOM(`
<body>
  <div id="container"></div>
  <script>
    class Component extends HTMLElement {
        connectedCallback() {
            this.attachShadow({ mode: "open" });
            this.shadowRoot.innerHTML = "<p>Hello world</p>";
        }
    }
    customElements.define('custom-component', Component);

    const container = document.querySelector("#container");

    Object.assign(container, {
        innerHTML: "<custom-component></custom-component>"
    })

    console.log(container.innerHTML); // <custom-component></custom-component>
    console.log(container.firstChild.shadowRoot.innerHTML); // <p>Hello world</p>
  </script>
</body>
`, { 
    runScripts: "dangerously" 
});

The illegal constructor probably is thrown by the original HTMLElement constructor, the changes mades on the branch should patch the constructor for each new window object. @tbranyen Do you have a complete reproduction example so I can try it locally?

Hi @pmdartus I'm not too sure yet what's causing my issues, but I wrote out some isolated code directly into your branch which worked perfectly:

const { JSDOM } = require('.');
const window = (new JSDOM()).window;
const { HTMLElement, customElements, document } = window;

class CustomElement extends HTMLElement {
  constructor() {
    super();

    console.log('constructed');
  }

  connectedCallback() {
    console.log('connected');
  }
}

customElements.define('custom-element', CustomElement);
document.body.appendChild(new CustomElement());
//constructed
//connected

{
  const window = (new JSDOM()).window;
  const { HTMLElement, customElements, document } = window;

  class CustomElement extends HTMLElement {
    constructor() {
      super();

      console.log('Constructed');
    }

    connectedCallback() {
      console.log('Connected');
    }
  }

  customElements.define('custom-element', CustomElement);
  document.body.appendChild(new CustomElement());
  //constructed
  //connected
}

This is effectively what my test system does, but breaks. So it may be something on my end.

Edit:

Ah okay I think I narrowed down where the problem is most likely occurring. I must be holding on to the initial HTMLElement constructor created. If I adjust the above code to reuse the constructor:

  // Inside the block, second component, reuse the HTMLElement.
  const { customElements, document } = window;

This will produce the following:

connected
/home/tbranyen/git/pmdartus/jsdom/lib/jsdom/living/helpers/create-element.js:643
        throw new TypeError("Illegal constructor");

Edit 2:

Found it:

  // Don't reuse the previously defined Element...
  global.HTMLElement = global.HTMLElement || jsdom.window.HTMLElement;

Noticing this thread is 4 years old, are web components supported or planned to be?

It would be nice to have web components in this, but as a alternative if anyone would like to know.... headless chrome can now be used in node to render/build the html sting file.

Noticing this thread is 4 years old, are web components supported or planned to be?

Its a work in progress as the spec is implemented piece by piece.

The polyfill at: https://github.com/WebReflection/document-register-element Works like a charm! My most sincere thanks to the author!

For those struggling with the same problem, just do:

npm install -D document-register-element

In your jest configuration set a setup file that will be run before all your tests:

{ "setupFilesAfterEnv": [ "./tests/setup.js" ] }

And finally, inside that file ('tests/setup.js'):

import 'document-register-element'

After doing this, registering and creating custom elements in jsdom via document.createElement('custom-component') works perfectly! Fragments don't seem to work, however. (I'm not using shadow dom, by the way).

@tebanep as you mentioned that polyfill is unsuitable for most Web Component work, if it doesn't support Shadow DOM then it's not really a comparison to what this is accomplishing.

@tebanep Thanks. Since i dont need shadow dom this is great info

Any hope that this will be implemented? Right now we are usign jsdom-wc with a lot bugs, but haven't have any better solution. My hope and pray on this topic.

@dknight I know jsdom-wc is pretty much a hack to get it kinda-sorta working. I published the module with significantly better compatibility under my personal npm scope. You can install it with:

npm install @tbranyen/[email protected] --save-dev

I use this now for all my JSDOM webcomponent needs until stable lands.

@tbranyen Did you unpublish your fork? I can't find it on npm.

@KingHenne dangit, looks like it ended up in our "enterprise" registry. I just published to the public npm. Sorry about that!

Don't @ me, but shouldn't we just test web ui code in a real browser e.g. with puppeteer. Shadow DOM/ Custom Elements support issue goes away then.

Don't post a comment if you don't want to be @'d @Georgegriff. That's a valid strategy, but it's slow and buggy in other ways since you're effectively doing IPC, yes even with puppeteer. When the browser dies, it's not obvious why in many cases. Just try and debug puppeteer issues in jest to get a taste of why it's not always the best idea.

Personally I'd rather keep testing synchronous and on the same thread. There's no reason why an isolated implementation of the specification shouldn't be a reasonable runtime for testing components. JSDOM is effectively a browser at this point, just not as stable as the big three.

The polyfill at: https://github.com/WebReflection/document-register-element Works like a charm! My most sincere thanks to the author!

For those struggling with the same problem, just do:

npm install -D document-register-element

In your jest configuration set a setup file that will be run before all your tests:

{ "setupFilesAfterEnv": [ "./tests/setup.js" ] }

And finally, inside that file ('tests/setup.js'):

import 'document-register-element'

After doing this, registering and creating custom elements in jsdom via document.createElement('custom-component') works perfectly! Fragments don't seem to work, however. (I'm not using shadow dom, by the way).

It works fine for me but the connectedCallback is never called, any idea?

@FaBeyyy same for me :(

@FaBeyyy @majo44 you have to append your component to a document ie. document.body.appendChild(...) for connectedCallback to get fired. By specs it’s being called when component is attached to a Dom.

JSDOM is effectively a browser at this point, just not as stable as the big three.

At this point, it’s more like the big two, because Microsoft is ditching theirs, which has been with them for as long as Windows.

@FaBeyyy @majo44 you have to append your component to a document ie. document.body.appendChild(...) for connectedCallback to get fired. By specs it’s being called when component is attached to a Dom.

thanks captain obvious but that is of course not the issue here. If I didn't know how the component lifecycle works I would probably not be trying to write tests 😄. Will create a repo showcase later when i find the time.

@FaBeyyy
So I found the setup which works for me. I had to add polyfill for MutationObserver. I'm using JSDOM for testing porpoise, with Jest, and the working setup is:

// package.json
{  ...
  "jest": {
    "transform": {
      "^.+\\.(mjs|jsx|js)$": "babel-jest"
    },
    "setupFiles": [
      "<rootDir>/node_modules/babel-polyfill/dist/polyfill.js",
      "<rootDir>/node_modules/mutationobserver-shim/dist/mutationobserver.min.js",
      "<rootDir>/node_modules/document-register-element/build/document-register-element.node.js"
    ]
  }
... 
}

//.bablerc
{
    "presets": [
        ["@babel/preset-env", { "modules": "commonjs"}]
    ]
}

@FaBeyyy
So I found the setup which works for me. I had to add polyfill for MutationObserver. I'm using JSDOM for testing porpoise, with Jest, and the working setup is:

// package.json
{  ...
  "jest": {
    "transform": {
      "^.+\\.(mjs|jsx|js)$": "babel-jest"
    },
    "setupFiles": [
      "<rootDir>/node_modules/babel-polyfill/dist/polyfill.js",
      "<rootDir>/node_modules/mutationobserver-shim/dist/mutationobserver.min.js",
      "<rootDir>/node_modules/document-register-element/build/document-register-element.node.js"
    ]
  }
... 
}

//.bablerc { "presets": [ ["@babel/preset-env", { "modules": "commonjs"}] ] }

Nice, Thanks!

@majo44 this is not required with latest jsdom. When working with Jest (which is using jsdom v11) you can just use updated environment: https://www.npmjs.com/package/jest-environment-jsdom-fourteen

@mgibas thanks, with jest-environment-jsdom-fourteen it is also works fine and mutation observer polyfill is not required (but version is 0.1.0, single commit package :) )

Is there a breakdown of which of the web components APIs are currently supported by JSDOM? Seems like shadow DOM is supported, but not custom elements (at least in the release branch/repo)?

npm install @tbranyen/[email protected] --save-dev

@tbranyen do you have the source code for your fork available somewhere? Would be curious to look at the diff 🙂

I'm using jest-environment-jsdom-fifteen like @majo44 suggested, and the babel-polyfill and document-register-element (see @mgibas answers). But I still get an error when I try to retrieve my web component shadow dom for tests.

el.shadowRoot is null with:

const el;
beforeEach(async() => {
  const tag= 'my-component'
  const myEl = document.createElement(tag);
  document.body.appendChild(myEl);
  await customElements.whenDefined(tag);
  await new Promise(resolve => requestAnimationFrame(() => resolve()));
  el = document.querySelector(tag);
}

it(() => {
  const fooContent = el.shadowRoot.querySelectorAll('slot[name=foo] > *');
})

Any idea of a workaround? FYI, it was already tested with Karma, Mocha, Chai & Jasmine.

Nothing special in my component:

customElements.define(
  'my-component',
  class extends HTMLElement {
    constructor() {
      super();

      const shadowRoot = this.attachShadow({ mode: 'open' });
      ...
  }
})

Edit

I did some debugging with jsdom 15.1.1 in order to better understand my issue.
Still, I don't understand why it's null here...

So, Element.shadowRoot is implemented since 88e72ef
https://github.com/jsdom/jsdom/blob/1951a19d8d40bc196cfda62a8dafa76ddda6a0d2/lib/jsdom/living/nodes/Element-impl.js#L388-L415

After document.createElement, this._shadowDom is ok at https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L403. And every element in the shadow dom is created (Element constructor called with the right values).

But when I call el.shadowDom immediately after document.body.appendChild(el) (https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L408), this. _shadowRoot is null!

Same thing after

 await customElements.whenDefined(tag);
 await new Promise(resolve => requestAnimationFrame(() => resolve()));

Or even if I use the following instead of document.

document.body.innerHTML = `
  <my-component id="fixture"></my-component>
`:

For reproduction, see:
https://github.com/noelmace/devcards/tree/jest

@nminhnguyen I guess you can find the source code of the fork made by @tbranyan here https://github.com/tbranyen/jsdom/tree/initial-custom-elements-impl

I'm trying to test web components made with lit-html and lit-element and I noticed this difference when creating the elements.

const myElem = new MyElem();

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) // exists and has the rendered markup

and when I use the document.createElement

const myElem = document.createElement('my-elem');

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) // null

For configuring jest I only use one polyfill that is : setupFiles: ['document-register-element']

Seems that the render method in myElem never gets called. Debugging a bit further I've discovered that the method initialize that is in lit-element never gets called.
So the 2nd example would work if I do

const myElem = document.createElement('my-elem');
myElem.initialize();

document.body.appendChild(myElem);
await myElem.updateComplete;

console.log(myElem.shadowRoot) //  exists and has the rendered markup

I have created an alternative DOM that supports web components. I first tried to make a PR, but the way JSDOM works made it hard for me to solve my needs there. You are free to use it or look at the code.

DOM:
https://www.npmjs.com/package/happy-dom

Jest environment:
https://www.npmjs.com/package/jest-environment-happy-dom

Looks awesome. Thank you.

On Mon, Oct 7, 2019, 1:08 AM capricorn86 notifications@github.com wrote:

I have created an alternative DOM that supports web components. I first
tried to make a PR, but the way JSDOM works made it hard for me to solve my
needs there. You are free to use it and/or look at the code.

DOM:
https://www.npmjs.com/package/happy-dom

Jest environment:
https://www.npmjs.com/package/jest-environment-happy-dom

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/jsdom/jsdom/issues/1030?email_source=notifications&email_token=ACQ5ZD5QUEITPND4SXWOHW3QNILSRA5CNFSM4A4G5SF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOO5ZA#issuecomment-538767076,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACQ5ZDYU465DXI4KHBQH4KTQNILSRANCNFSM4A4G5SFQ
.

@capricorn86
Your work makes my Test environment simple, thanks!
https://github.com/EasyWebApp/WebCell/tree/v2/MobX

@capricorn86
Your work makes my Test environment simple, thanks!
https://github.com/EasyWebApp/WebCell/tree/v2/MobX

Thank you @TechQuery!

Looks awesome. Thank you.
…
On Mon, Oct 7, 2019, 1:08 AM capricorn86 @.*> wrote: I have created an alternative DOM that supports web components. I first tried to make a PR, but the way JSDOM works made it hard for me to solve my needs there. You are free to use it and/or look at the code. DOM: https://www.npmjs.com/package/happy-dom Jest environment: https://www.npmjs.com/package/jest-environment-happy-dom — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1030?email_source=notifications&email_token=ACQ5ZD5QUEITPND4SXWOHW3QNILSRA5CNFSM4A4G5SF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOO5ZA#issuecomment-538767076>, or mute the thread https://github.com/notifications/unsubscribe-auth/ACQ5ZDYU465DXI4KHBQH4KTQNILSRANCNFSM4A4G5SFQ .

Thank you @motss!

Is there a breakdown of which of the web components APIs are currently supported by JSDOM? Seems like shadow DOM is supported, but not custom elements (at least in the release branch/repo)?

I'd be interested in this as well :)