Jinja: Impossible to iterate over object with "items" element

Created on 9 Jan 2017  ·  10Comments  ·  Source: pallets/jinja

A server is returning this JSON:
{
"items": [1,2,3]
...
}

An attempt to iterate over 'items' produces:
{% for el in data.items %}
....
TypeError: 'builtin_function_or_method' object is not iterable

Above happens because Python dict has a built-in function "items()".

As a temp. work-around, I installed a hook and rename keys in the JSON response from "items" to "__items" then pass that to Jinja2. At that point, I'm iterating over the renamed attribute fine:
{% for el in data.__items %}

Ideally, Jinja2 should ignore built-in functions in for-loop or if-statement unless specifically invoked with a function() notation.

Most helpful comment

You can use {% for el in data['items'] %} to prioritize the item. This is also covered in the documentation. http://jinja.pocoo.org/docs/2.9/templates/#variables

All 10 comments

You can use {% for el in data['items'] %} to prioritize the item. This is also covered in the documentation. http://jinja.pocoo.org/docs/2.9/templates/#variables

It will fail if the JSON response occasionally does not include the 'items' (say it was an optional attribute of the response). In that case, {% for el in data['items'] %} will also produce:
TypeError: 'builtin_function_or_method' object is not iterable

...since items is always defined (as a builtin function).

And this is a bug, IMO. The data['items'] should never collide with the builtins. In other words, if you want to cleanly iterate an optional "items" attribute you are forced to rename it. Ideally, that shouldn't be the case.

Why would the JSON response not include items? I'm not entirely sure I see the problem here. Surely you know ahead of time what your data is. You can also in templates check for keys with the in operator on dictionaries already.

Changing this behavior is completely out of the question because thousands of templates would break if we remove this attribute/item support.

I'm not suggesting to drop anything, just to make it more strict.

The problem today is that when you use the explicit lookup notation: data['items'] it finds a builtin dictionary function and it shouldn't. Python doesn't do it, nor should jinja. If one needs to call the builtin (or any function for that matter), you have the calling notation for it: data.items()

Not sure if it would be very clear to special-case dict.items there. Especially since it'd still fail for custom dict-like types or maybe even OrderedDict. I think it's better to foo['items'] in those cases. It's not super pretty but consistent behavior.

The only possible "fix/hack" I could imagine to hancle this in a nicer way is checking whether a plain attribute (i.e. not a function call) is used in an iteration and you are trying to iterate over something that is not iterable but callable. In this case trying the item would be very unlikely to break any existing code. However, the implementation for such behavior in Jinja would probably be extremely awful ;)

Why is this a special case? I'm asking for Jinja to behave exactly like Python does. Specifically, foo['items'] should yield None if there is no user-inserted key "items" in it. Today, Jinja falls back to finding the built in function of the same name. So simply do what Python does:

>>> print {}.get('items')
None

Btw, I also believe that the alternative notation "foo.items" should be handled just fine as well, since you need to use "()" to call a callable, but if you require some backwards compatibility for foo.items to either find a key or call a function automatically, then I guess that's too bad.

The Jinja logic for both ['x'] and .x is this:

  • Check if the item/attr (whatever your syntax gets in Python) exists, if yes use that
  • Check if the attr/item (the "opposite" of what your syntax would do in Python) exists, if yes use that
  • Fail if neither exist

So x.items checks for the attribute and since one exists it uses that. This logic works like this no matter the contest. Improving this would require the attribute lookup logic to change depending on its context. And even then it would result in awkward situations like this code that would work:

{% for x in mydict.items %}

vs this code which would not work

{% set items = mydict.items %}
{% for x in items %}

And keeping special handling in this case would simply be impossible without keeping track of where a variable came from.

@slisznia you are essentially asking for Jinja2 to completely change the attribute lookup system which has been in place for about 7 years at this point.

Ah, I didn't even see the suggestion of foo['bar'] not doing an attribute lookup. That's a bad idea for sure! What's wrong with you simply using for item in data.get('items', []) ?

@ThiefMaster the mydict.get('items', []) works of course, and thanks for pointing out another method than renaming keys.

Was this page helpful?
0 / 5 - 0 ratings