I have an array of true/false/undefined values which I am rendering as a list of checkboxes.
When changing an array element to or from true, the list of checkboxes is re-rendered with the following (index+1) checkbox inheriting the change along with the changed checkbox.
Code:
{{#each range as |value idx|}}
<label><input type="checkbox" checked={{value}} {{action makeChange idx on="change"}}>{{idx}}: {{value}}</label><br/>
{{/each}}
When I use {{#each range key="@index" as |value idx|}}
it works correctly.
Twiddle: https://ember-twiddle.com/6d63548f35f99da19cee9f58fb64db59
@andrewtimberlake seems like using the {{#each range key="@index" as |value idx|}}
does workaround the issue.
But does seem like a bug, the key
is for a different purpose, https://www.emberjs.com/api/ember/release/classes/Ember.Templates.helpers/methods/if?anchor=each
I think I know what's going on here. It's a 🍝 of mess but I'll try to describe it. A lot of edge cases (border line user-error) contributed to this, and I'm not really sure what is/is not a bug, what and how to fix any of these.
First of all, I need to describe what the key
parameter does in {{#each}}
. TL;DR it's trying to determine when and if it would make sense to reuse the existing DOM, vs just creating the DOM from scratch.
For our purpose, let's accept as a given that "touching DOM" (e.g. updating the content of a text node, an attribute, adding or removing content etc) is expensive and is to be avoided as much as possible.
Let's focus on a rather simple piece of template:
<ul>
{{#each this.names as |name|}}
<li>{{name.first}} {{to-upper-case name.last}}</li>
{{/each}}
</ul>
If this.names
is...
[
{ first: "Yehuda", last: "Katz" },
{ first: "Tom", last: "Dale" },
{ first: "Godfrey", last: "Chan" }
]
Then you will get...
<ul>
<li>Yehuda KATZ</li>
<li>Tom DALE</li>
<li>Godfrey CHAN</li>
</ul>
So far so good.
Now what if we append { first: "Andrew", last: "Timberlake" }
to the list? We would expect the template to produce the following DOM:
<ul>
<li>Yehuda KATZ</li>
<li>Tom DALE</li>
<li>Godfrey CHAN</li>
<li>Andrew TIMBERLAKE</li>
</ul>
But _how_?
The most naive way to implement the {{#each}}
helper would clear the all content of the list every time the content of the list changes. To do this you would need perform _at least_ 23 operations:
<li>
nodes<li>
nodesto-upper-case
helper 4 timesThis seems... very unnecessary and expensive. We _know_ the first three items didn't change, so it would be nice if we could just skip the work for those rows.
A better implementation would be to try to reuse the existing rows and don't do any unnecessary updates. One idea would be to simply match up the rows with their positions in the templates. This is essentially what key="@index"
does:
{ first: "Yehuda", last: "Katz" }
with the first row, <li>Yehuda KATZ</li>
:to-upper-case
helper, and therefore we know the output of that helper ("KATZ") _also_ didn't change, so nothing to do here<li>
nodeto-upper-case
helper ("Timberlake" -> "TIMBERLAKE")So, with this implementation, we reduced the total number of operations from 23 to 5 (👋 hand-waving over the cost of the comparisons, but for our purpose, we are assuming they are relatively cheap compared to the rest). Not bad.
But now, what would happen if, instead of _appending_ { first: "Andrew", last: "Timberlake" }
to the list, we _prepended_ it instead? We would expect the template to produce the following DOM:
<ul>
<li>Andrew TIMBERLAKE</li>
<li>Yehuda KATZ</li>
<li>Tom DALE</li>
<li>Godfrey CHAN</li>
</ul>
But _how_?
{ first: "Andrew", last: "Timberlake" }
with the first row, <li>Yehuda KATZ</li>
:to-upper-case
helper{ first: "Yehuda", last: "Katz" }
with the second row, <li>Tom DALE</li>
, another 3 operation{ first: "Tom", last: "Dale" }
with the second row, <li>Godfrey CHAN</li>
, another 3 operation<li>
nodeto-upper-case
helper ("Chan" -> "CHAN")That's 14 operations. Ouch!
That seemed unnecessary, because conceptually, whether we are prepending or appending, we are still only changing (inserting) a single object in the array. Optimally, we should be able to handle this case just as well as we did in the append scenario.
This is where key="@identity"
comes in. Instead of relying on the _order_ of the elements in the array, we use their JavaScript object identity (===
):
===
) the first object { first: "Andrew", last: "Timberlake" }
. Since nothing was found, insert (prepend) a new row:<li>
nodeto-upper-case
helper ("Timberlake" -> "TIMBERLAKE")===
) the second object { first: "Yehuda", last: "Katz" }
. Found <li>Yehuda KATZ</li>
:to-upper-case
helper, and therefore we know the output of that helper ("KATZ") _also_ didn't change, so nothing to do hereWith that, we are back to the optimal 5 operations.
Again this is 👋 hand-waving over the comparisons and book-keeping costs. Indeed, those are not free either, and in this very simple example, they may not be worth it. But imagine the list is big and each row invokes a complicated component (with lots of helpers, computed properties, sub-components, etc). Imagine the LinkedIn news-feed, for example. If we don't match up the right rows with the right data, your components' arguments can potentially churn a lot and causes much more DOM updates than you may otherwise expect. There are also issues with matching up the wrong DOM elements and loosing DOM state, such as cursor position and text selection state.
Overall, the extra comparison and book-keeping cost is easily worth most of the time in a real-world app. Since the key="@identity"
is the default in Ember and it works well for almost all cases, you typically won't have to worry about setting the key
argument when using {{#each}}
.
But wait, there is a problem. What about this case?
const YEHUDA = { first: "Yehuda", last: "Katz" };
const TOM = { first: "Tom", last: "Dale" };
const GODFREY = { first: "Godfrey", last: "Chan" };
this.list = [
YEHUDA,
TOM,
GODFREY,
TOM, // duplicate
YEHUDA, // duplicate
YEHUDA, // duplicate
YEHUDA // duplicate
];
The problem here is that the same object _could_ appear multiple times in the same list. This breaks our naive @identity
algorithm, specifically the part where we said "Find an existing row whose data matches (===
) ..." – this only works if the data to DOM relationship is 1:1, which is not true in this case. This may seem unlikely in practice, but as a framework, we have to handle it.
To avoid this, we use sort of a hybrid approach to handle these collisions. Internally, the keys-to-DOM mapping looks something like this:
"YEHUDA" => <li>Yehuda...</li>
"TOM" => <li>Tom...</li>
"GODFREY" => <li>Godfrey...</li>
"TOM-1" => <li>Tom...</li>
"YEHUDA-1" => <li>Yehuda...</li>
"YEHUDA-2" => <li>Yehuda...</li>
"YEHUDA-3" => <li>Yehuda...</li>
For the most part, this _is_ pretty rare, and when it does come up, this works Good Enough™ most of the time. If, for some reason, this doesn't work, you can always use a key path (or the even more advanced keying mechanism in RFC 321).
After all that talk we are now ready to look at the scenario in the Twiddle.
Essentially, we started with this list: [undefined, undefined, undefined, undefined, undefined]
.
Unrelated note:
Array(5)
is _not_ the same thing as[undefined, undefined, undefined, undefined, undefined]
. It produces a "holey array" which is something you should avoid in general. However, it is unrelated to this bug, because when accessing the "holes" you indeed get anundefined
back. So for our _very narrow_ purpose only, they are the same.
Since we didn't specify the key, Ember uses @identity
by default. Further, since they are collisions, we ended up with something like this:
"undefined" => <input ...> 0: ...,
"undefined-1" => <input ...> 1: ...,
"undefined-2" => <input ...> 2: ...,
"undefined-3" => <input ...> 3: ...,
"undefined-4" => <input ...> 4: ...
Now, let's say we click the first check box:
{{action}}
modifier and re-dispatched to the makeChange
method[true, undefined, undefined, undefined, undefined]
.How is the DOM updated?
===
) the first object true
. Since nothing was found, insert (prepend) a new row <input checked=true ...>0: true...
===
) the second object undefined
. Found <input ...>0: ...
(previously the FIRST row):{{idx}}
text node to 1
===
) the third object undefined
. Since this is the second time we saw undefined
, the internal key is undefined-1
, so we found <input ...>1: ...
(previously the SECOND row):{{idx}}
text node to 2
undefined-2
and undefined-3
undefined-4
row (since there are one fewer undefined
in the array after the update)So this explains how we got the output you had in the twiddle. Essentially all the DOM rows shifted down by one, and a new one was inserted at the top, while the {{idx}}
is updated for the rest.
The really unexpected part is 2.2. Even though the first checkbox (the one that was clicked on) was shifted down by one row to the second position, you would probably have expected Ember to its checked
property has changed to true
, and since its bound value is undefined, you may expect Ember to change it back to false
, thus unchecking it.
But this is not how it works. As mentioned in the beginning, accessing DOM is expensive. This includes _reading_ from DOM. If, on every update, we had to read the latest value from the DOM for our comparisons, it would pretty much defeat the purpose of our optimizations. Therefore, in order to avoid that, we remembered the last value that we had written to the DOM, and compare the current value against the cached value without having to read it back from the DOM. Only when there is a difference do we write the new value to the DOM (and cache it for next time). This is the sense in which we kind of share the same "virtual DOM" approach, but we only do it at the leaf nodes, not virtualizing the "tree-ness" of the whole DOM.
So, TL;DR, "binding" the checked
property (or the value
property of a text field, etc) doesn't really work the way you expect. Imagine if you rendered <div>{{this.name}}</div>
and you manually updated the textContent
of the div
element using jQuery
or with the chrome inspector. You wouldn't have expected Ember to notice that and update this.name
for you. This is basically the same thing: since the update to the checked
property happened outside of Ember (through the browser's default behavior for the checkbox), Ember is not going to know about that.
This is why the {{input}}
helper exists. It has to register the relevant event listeners on the underlying HTML element and reflect the operations into the appropriate property change, so the interested parties (e.g. the rendering layer) can be notified.
I'm not sure where that leaves us. I understand why this is surprising, but I'm inclined to say this is a series of unfortunate user errors. Perhaps we should be linting against binding these properties on input elements?
Relevant code:
https://github.com/emberjs/ember.js/blob/c24bc23e4139c90c8d8d96c4234d9c0c19e5c594/packages/@ember/-internals/glimmer/lib/utils/iterable.ts#L390-L391
https://github.com/emberjs/ember.js/blob/c24bc23e4139c90c8d8d96c4234d9c0c19e5c594/packages/@ember/-internals/glimmer/lib/utils/iterable.ts#L436-L445
https://github.com/emberjs/ember.js/blob/c24bc23e4139c90c8d8d96c4234d9c0c19e5c594/packages/@ember/-internals/glimmer/lib/utils/iterable.ts#L451-L466
@chancancode - thanks for the amazing explanation. Does that mean that <input ... >
should never be used but only {{input ...}}
in order to prevent all errors like that?
@boris-petrov there may be some limited cases where it is acceptable.. like a readonly textfield for "copy this url to your clipboard" kind of thing, or you _can_ use the input element + {{action}}
to intercept the DOM event and reflect the property updates manually (which is what the twiddle tried to do, except it also ran into the @identity
collision), but yes at some point you are just re-implementing {{input}}
and handling all the edge-cases it already handled for you. So I think it's _probably_ fair to say you should just use {{input}}
most, if not all, of the time.
However, that still wouldn't have "fixed" this case where there are collisions with the keys. See https://ember-twiddle.com/0f2369021128e2ae0c445155df5bb034?openFiles=templates.application.hbs%2C
Which is why I said, I'm 100% not sure what to do about this. On one hand I agree it is surprising and unexpected, on the other hand, this kind of collision is pretty rare in real apps and it is why the "key" argument is customizable (this is a case where the default "@identity" key function is not Good Enough™, which is why that feature exists).
@chancancode - this reminds me of another issue that I opened some time ago. Do you think there is something similar there? The answer that I received there (about the need to use replace
instead of set
when setting array elements) still seems strange to me.
@boris-petrov I don't think it is related
hi, we use sortablejs for draggable list with ember . pls check this demo for reproduce each problem .
step:
you can see the dragged item stay in dom tree .
but, if drag item to other position(not last item) , it seems that work well .
Most helpful comment
I think I know what's going on here. It's a 🍝 of mess but I'll try to describe it. A lot of edge cases (border line user-error) contributed to this, and I'm not really sure what is/is not a bug, what and how to fix any of these.
Major 🔑
First of all, I need to describe what the
key
parameter does in{{#each}}
. TL;DR it's trying to determine when and if it would make sense to reuse the existing DOM, vs just creating the DOM from scratch.For our purpose, let's accept as a given that "touching DOM" (e.g. updating the content of a text node, an attribute, adding or removing content etc) is expensive and is to be avoided as much as possible.
Let's focus on a rather simple piece of template:
If
this.names
is...Then you will get...
So far so good.
Appending an item to the list
Now what if we append
{ first: "Andrew", last: "Timberlake" }
to the list? We would expect the template to produce the following DOM:But _how_?
The most naive way to implement the
{{#each}}
helper would clear the all content of the list every time the content of the list changes. To do this you would need perform _at least_ 23 operations:<li>
nodes<li>
nodesto-upper-case
helper 4 timesThis seems... very unnecessary and expensive. We _know_ the first three items didn't change, so it would be nice if we could just skip the work for those rows.
🔑 @index
A better implementation would be to try to reuse the existing rows and don't do any unnecessary updates. One idea would be to simply match up the rows with their positions in the templates. This is essentially what
key="@index"
does:{ first: "Yehuda", last: "Katz" }
with the first row,<li>Yehuda KATZ</li>
:1.1. "Yehuda" === "Yehuda", nothing to do
1.2. (the space does not contain dynamic data so no comparison needed)
1.3. "Katz" === "Katz", since helpers are "pure", we know we won't have to re-invoke the
to-upper-case
helper, and therefore we know the output of that helper ("KATZ") _also_ didn't change, so nothing to do here3.1. Insert a
<li>
node3.2. Insert a text node ("Andrew")
3.3. Insert a text node (the space)
3.4. Invoke the
to-upper-case
helper ("Timberlake" -> "TIMBERLAKE")3.5. Insert a text node ("TIMBERLAKE")
So, with this implementation, we reduced the total number of operations from 23 to 5 (👋 hand-waving over the cost of the comparisons, but for our purpose, we are assuming they are relatively cheap compared to the rest). Not bad.
Prepending an item to the list
But now, what would happen if, instead of _appending_
{ first: "Andrew", last: "Timberlake" }
to the list, we _prepended_ it instead? We would expect the template to produce the following DOM:But _how_?
{ first: "Andrew", last: "Timberlake" }
with the first row,<li>Yehuda KATZ</li>
:1.1. "Andrew" !== "Yehuda", update the text node
1.2. (the space does not contain dynamic data so no comparison needed)
1.3. "Timberlake" !== "Katz", reinvoke the
to-upper-case
helper1.4. Update the text node from "KATZ" to "TIMBERLAKE"
{ first: "Yehuda", last: "Katz" }
with the second row,<li>Tom DALE</li>
, another 3 operation{ first: "Tom", last: "Dale" }
with the second row,<li>Godfrey CHAN</li>
, another 3 operation3.1. Insert a
<li>
node3.2. Insert a text node ("Godfrey")
3.3. Insert a text node (the space)
3.4. Invoke the
to-upper-case
helper ("Chan" -> "CHAN")3.5. Insert a text node ("CHAN")
That's 14 operations. Ouch!
🔑 @identity
That seemed unnecessary, because conceptually, whether we are prepending or appending, we are still only changing (inserting) a single object in the array. Optimally, we should be able to handle this case just as well as we did in the append scenario.
This is where
key="@identity"
comes in. Instead of relying on the _order_ of the elements in the array, we use their JavaScript object identity (===
):===
) the first object{ first: "Andrew", last: "Timberlake" }
. Since nothing was found, insert (prepend) a new row:1.1. Insert a
<li>
node1.2. Insert a text node ("Andrew")
1.3. Insert a text node (the space)
1.4. Invoke the
to-upper-case
helper ("Timberlake" -> "TIMBERLAKE")1.5. Insert a text node ("TIMBERLAKE")
===
) the second object{ first: "Yehuda", last: "Katz" }
. Found<li>Yehuda KATZ</li>
:2.1. "Yehuda" === "Yehuda", nothing to do
2.2. (the space does not contain dynamic data so no comparison needed)
2.3. "Katz" === "Katz", since helpers are "pure", we know we won't have to re-invoke the
to-upper-case
helper, and therefore we know the output of that helper ("KATZ") _also_ didn't change, so nothing to do hereWith that, we are back to the optimal 5 operations.
Scaling Up
Again this is 👋 hand-waving over the comparisons and book-keeping costs. Indeed, those are not free either, and in this very simple example, they may not be worth it. But imagine the list is big and each row invokes a complicated component (with lots of helpers, computed properties, sub-components, etc). Imagine the LinkedIn news-feed, for example. If we don't match up the right rows with the right data, your components' arguments can potentially churn a lot and causes much more DOM updates than you may otherwise expect. There are also issues with matching up the wrong DOM elements and loosing DOM state, such as cursor position and text selection state.
Overall, the extra comparison and book-keeping cost is easily worth most of the time in a real-world app. Since the
key="@identity"
is the default in Ember and it works well for almost all cases, you typically won't have to worry about setting thekey
argument when using{{#each}}
.Collisions 💥
But wait, there is a problem. What about this case?
The problem here is that the same object _could_ appear multiple times in the same list. This breaks our naive
@identity
algorithm, specifically the part where we said "Find an existing row whose data matches (===
) ..." – this only works if the data to DOM relationship is 1:1, which is not true in this case. This may seem unlikely in practice, but as a framework, we have to handle it.To avoid this, we use sort of a hybrid approach to handle these collisions. Internally, the keys-to-DOM mapping looks something like this:
For the most part, this _is_ pretty rare, and when it does come up, this works Good Enough™ most of the time. If, for some reason, this doesn't work, you can always use a key path (or the even more advanced keying mechanism in RFC 321).
Back to the "🐛"
After all that talk we are now ready to look at the scenario in the Twiddle.
Essentially, we started with this list:
[undefined, undefined, undefined, undefined, undefined]
.Since we didn't specify the key, Ember uses
@identity
by default. Further, since they are collisions, we ended up with something like this:Now, let's say we click the first check box:
{{action}}
modifier and re-dispatched to themakeChange
method[true, undefined, undefined, undefined, undefined]
.How is the DOM updated?
===
) the first objecttrue
. Since nothing was found, insert (prepend) a new row<input checked=true ...>0: true...
===
) the second objectundefined
. Found<input ...>0: ...
(previously the FIRST row):2.1. Update the
{{idx}}
text node to1
2.2. Otherwise, as far as Ember can tell, nothing else has changed in this row, nothing else to do
===
) the third objectundefined
. Since this is the second time we sawundefined
, the internal key isundefined-1
, so we found<input ...>1: ...
(previously the SECOND row):3.1. Update the
{{idx}}
text node to2
3.2. Otherwise, as far as Ember can tell, nothing else has changed in this row, nothing else to do
undefined-2
andundefined-3
undefined-4
row (since there are one fewerundefined
in the array after the update)So this explains how we got the output you had in the twiddle. Essentially all the DOM rows shifted down by one, and a new one was inserted at the top, while the
{{idx}}
is updated for the rest.The really unexpected part is 2.2. Even though the first checkbox (the one that was clicked on) was shifted down by one row to the second position, you would probably have expected Ember to its
checked
property has changed totrue
, and since its bound value is undefined, you may expect Ember to change it back tofalse
, thus unchecking it.But this is not how it works. As mentioned in the beginning, accessing DOM is expensive. This includes _reading_ from DOM. If, on every update, we had to read the latest value from the DOM for our comparisons, it would pretty much defeat the purpose of our optimizations. Therefore, in order to avoid that, we remembered the last value that we had written to the DOM, and compare the current value against the cached value without having to read it back from the DOM. Only when there is a difference do we write the new value to the DOM (and cache it for next time). This is the sense in which we kind of share the same "virtual DOM" approach, but we only do it at the leaf nodes, not virtualizing the "tree-ness" of the whole DOM.
So, TL;DR, "binding" the
checked
property (or thevalue
property of a text field, etc) doesn't really work the way you expect. Imagine if you rendered<div>{{this.name}}</div>
and you manually updated thetextContent
of thediv
element usingjQuery
or with the chrome inspector. You wouldn't have expected Ember to notice that and updatethis.name
for you. This is basically the same thing: since the update to thechecked
property happened outside of Ember (through the browser's default behavior for the checkbox), Ember is not going to know about that.This is why the
{{input}}
helper exists. It has to register the relevant event listeners on the underlying HTML element and reflect the operations into the appropriate property change, so the interested parties (e.g. the rendering layer) can be notified.I'm not sure where that leaves us. I understand why this is surprising, but I'm inclined to say this is a series of unfortunate user errors. Perhaps we should be linting against binding these properties on input elements?