Django-filter: [2.1.0] BaseCSVWidget.value_from_datadict — AttributeError: 'list' object has no attribute 'split'

Created on 12 Jul 2019  ·  15Comments  ·  Source: carltongibson/django-filter

I ran into …

Traceback (most recent call last):
  [..]
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'

and found that the code at …
https://github.com/carltongibson/django-filter/blob/1f47e36b614724a8735e0457fa511dcaf5448481/django_filters/widgets.py#L195-L202
… is not robust with regard to super().value_from_datadict(data, files, name) returning a list. I need to debug this more but maybe you already know what's going on here?

Most helpful comment

I just had this error and its explanation is buried in this thread, so to recap and save a long search for future readers:

ManyToMany fields declared in viewsets' filterset_fields must not contain in

A warning at runtime would be very appreciated.

All 15 comments

Hi @moseb. Can you paste the FilterSet code that you're using? The last few lines of the trace back are sufficient to determine why you'd hit this exception.

Full traceback:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/wsgiref/handlers.py", line 137, in run
    self.result = application(self.environ, self.start_response)
  File "/usr/local/lib/python3.7/site-packages/django/contrib/staticfiles/handlers.py", line 65, in __call__
    return self.application(environ, start_response)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/wsgi.py", line 141, in __call__
    response = self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 75, in get_response
    response = self._middleware_chain(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/utils/deprecation.py", line 94, in __call__
    response = response or self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django_global_request/middleware.py", line 15, in __call__
    return self.get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 36, in inner
    response = response_for_exception(request, exc)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 90, in response_for_exception
    response = handle_uncaught_exception(request, get_resolver(get_urlconf()), sys.exc_info())
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 145, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 143, in _get_response
    response = response.render()
  File "/usr/local/lib/python3.7/site-packages/django/template/response.py", line 106, in render
    self.content = self.rendered_content
  File "/usr/local/lib/python3.7/site-packages/rest_framework/response.py", line 72, in rendered_content
    ret = renderer.render(self.data, accepted_media_type, context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 733, in render
    context = self.get_context(data, accepted_media_type, renderer_context)
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 710, in get_context
    'filter_form': self.get_filter_form(data, view, request),
  File "/usr/local/lib/python3.7/site-packages/rest_framework/renderers.py", line 642, in get_filter_form
    html = backend().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/rest_framework_filters/backends.py", line 52, in to_html
    return super().to_html(request, queryset, view)
  File "/usr/local/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 105, in to_html
    return template.render(context, request)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 199, in render
    c = self.get_render(context).flatten()
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/templatetags/crispy_forms_tags.py", line 118, in get_render
    actual_form.form_html = helper.render_layout(actual_form, node_context, template_pack=self.template_pack)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/helper.py", line 308, in render_layout
    template_pack=template_pack
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 140, in render
    return self.get_rendered_fields(form, form_style, context, template_pack, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in get_rendered_fields
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/layout.py", line 104, in <genexpr>
    for field in self.fields
  File "/usr/local/lib/python3.7/site-packages/crispy_forms/utils.py", line 148, in render_field
    html = template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 309, in render
    return nodelist.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
    bit = node.render_annotated(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
    return self.render(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 302, in render
    match = condition.eval(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 876, in eval
    return self.value.resolve(context, ignore_failures=True)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 671, in resolve
    obj = self.var.resolve(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 796, in resolve
    value = self._resolve_lookup(context)
  File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 837, in _resolve_lookup
    current = getattr(current, bit)
  File "/usr/local/lib/python3.7/site-packages/django/forms/boundfield.py", line 74, in errors
    return self.form.errors.get(self.name, self.form.error_class())
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 180, in errors
    self.full_clean()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 381, in full_clean
    self._clean_fields()
  File "/usr/local/lib/python3.7/site-packages/django/forms/forms.py", line 393, in _clean_fields
    value = field.widget.value_from_datadict(self.data, self.files, self.add_prefix(name))
  File "/usr/local/lib/python3.7/site-packages/django_filters/widgets.py", line 201, in value_from_datadict
    return value.split(',')
AttributeError: 'list' object has no attribute 'split'
[12/Jul/2019 13:49:06] "GET /XXXXXXXX/?ordering=first_name HTTP/1.1" 500 59

It's from browsing a listing where django-rest-framework generates the code.

Hi @moseb. Can you paste the FilterSet code that you're using?

To be honest, I haven't found any in that area of the code, yet. It might be all by DRF. Not sure yet.
I'm aware my bug report is not ideal but it's the best I have right now. Sorry!

Your API view should have either a filterset_class or filterset_fields, or if you're using an older version of django-filter, filter_class and filter_fields.

I found use of filter_class — so I guess you just helped me find that the migration to django-filter 2.0 was not properly done to its end. Cool!
Is there something stopping django-filter from detecting pre-2.x leftovers and warning about them, conceptually? Does it warn but I missed it? Would code warning about these be a welcome addition for a pull request?

v2.0 removed a lot of the existing deprecation warnings, so you might try running your test suite against v1.0 and then v1.1 of django-filter.

Assuming this will resolve itself when you update. Come back if not.

There is reproducible demo of the issue now at https://github.com/moseb/django-filter-issue-1103-demo. Please consider re-opening this issue. Thank you!

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

Hi!

Ah, this is related to ModelMultipleChoiceFilter not being compatible with the CSV mixin, as well as the in lookup. Two separate but related issues here.

Is there existing tickets on GitHub for these issues?

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

You shouldn't need an in lookup for m2m fields, since the multiple choice filter already provides similar behavior.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

PS: Can we re-open this ticket?

It sounds like making BaseCSVWidget.value_from_datadict more robust as mentioned above alone will not fix the issue? (I don't fully understand the issue yet.)

Possibly. If the value is already a list, it should just go ahead and return that list. That said, I don't think it makes sense to mix the CSV-behavior with the SelectMultiple widget.

in allows checking against multiple values over exact. If I cannot use ìn, how do I check for multiple values?

The ModelMultipleChoiceFilter constructs an OR query from Q objects. So, if you have a query string like /api/mymodel?m2m=a&m2m=b, you would end up with a filter call like

MyModel.objects.filter(Q(m2m='a') | Q(m2m='b'))

With the 2.1 release, the lookup_expr is now applied to each Q object, so you'd end up with

MyModel.objects.filter(Q(m2m__in='a') | Q(m2m__in='b'))

The above is invalid and would break, however it makes sense for contains and other lookups that are intended to work with single values.

So, in short, all you need to do here is use the exact lookup for your m2m field.


I think the todo here is:

  • Make BaseCSVWidget compatible with SelectMultiple by returning the value if it's already a list. Or if it shouldn't be compatible, we should at least provide a useful error on init.
  • Fix Meta.fields for m2m. Generating an in lookup for ModelMultipleChoiceFilter is nonsensical. We should also test what happens for other lookups like isnull, which expects a boolean.

Yeah, this still doesn't look like a bug but how it all works is under documented I guess...

I just had this error and its explanation is buried in this thread, so to recap and save a long search for future readers:

ManyToMany fields declared in viewsets' filterset_fields must not contain in

A warning at runtime would be very appreciated.

@marcosox Good idea. Happy to look at a PR adding that!

Was this page helpful?
0 / 5 - 0 ratings