Django-rest-framework: Nested serializer does not receive view context

Context is not (cannot be) passed to the fields (or nested serializer) at the point of init, because they are initialized when they are declared on the parent serializer...

class MySerializer(Serializer):
    child = ChildSerializer()  <--- We've just initialized this.

Best is to perform any behavior like that in MySerializer.__init__, eg...

def __init__(self, *args, **kwargs):
    super(MySerializer, self).__init__(*args, **kwargs)
    self.fields['child'].default = ...

At that point you _do_ have access to the context.

If you really want you can perform some actions at the point the point the field becomes bound to its parent...

class ChildSerializer(Serializer):
    def bind(self, *args, **kwargs):
        super(ChildSerializer, self).bind(*args, **kwargs)
        # do stuff with self.context.

This should be considered private API, and the parent __init__ style listed above should be preferred.

Yea, I understand what you explain. That's why I think it is a bit tricky. That's why my workaround was to create nested serializer in the parent's __ini__. But nested serializer __init__ called once every new serializer instance is created, since it uses deepcopy, but we cannot sneak parent instance in it...
So, as I can see in v2, field had initialize method, which was perfect to init defaults, based on parent serializer.

So, it seems bind should be good.

This should be considered private API, and the parent __init__ style listed above should be preferred.

But since we just can't use __init__, because we cannot access all available environment, created by view at the moment, we should have public API method, which is called on field(serializer) when we have all things available. Like initialize was.

Does it work for DRF 2?

it's a pity context is no longer propagated to nested serializers.

it's a pity context is no longer propagated to nested serializers.

Where did you get that ? Afaik, context _is_ propagated to the nested serializers.

@xordoquy thats strange – I've made a testcase against latest drf and I cannot reproduce issue :) moreover I cannot reproduce it agains my own code. Problem solved.

@xiaohanyu actually I've found this problem once again. And it's kinda strange:

>>> serializer
Out[12]: 
# Note there is context passed in constructor
CouponSerializer(<Coupon: Coupon Coupon #0 for offer Offer #0000>, context={'publisher': <User: John ZusPJRryIzYZ>}):
    id = IntegerField(label='ID', read_only=True)
    title = CharField(max_length=100, required=True)
    short_title = CharField(max_length=30)
    offer_details = OfferLimitedSerializer(read_only=True, source='offer'):
        id = IntegerField(label='ID', read_only=True)
        title = CharField(help_text='The Offer Title will be used for the Network, Advertisers and Publishers to identify the specific offer within the application.', read_only=True)       
        status_name = CharField(read_only=True, source='get_status_display')
        is_access_limited = SerializerMethodField()    
    exclusive = BooleanField(required=False)    
    categories_details = OfferCategorySerializer(many=True, read_only=True, source='categories'):
        id = IntegerField(read_only=True)
        category = CharField(read_only=True, source='code')
        category_name = CharField(read_only=True, source='get_code_display')    

# all fields except one got context propagated
>>> [f.field_name for f in serializer.fields.values() if not f.context]
Out[20]: 
['offer_details']

# offer_details is no different than categories_details, both are nested, read only, model serializers..
>>> serializer.fields['categories_details'].context
Out[21]: 
{'publisher': <User: John ZusPJRryIzYZ>}

# lets look inside problematic serializer
>>> offer_details = serializer.fields['offer_details']
>>> offer_details
Out[25]: 

OfferLimitedSerializer(read_only=True, source='offer'):
    id = IntegerField(label='ID', read_only=True)
    title = CharField(help_text='The Offer Title will be used for the Network, Advertisers and Publishers to identify the specific offer within the application.', read_only=True)    
    status_name = CharField(read_only=True, source='get_status_display')
    url = SerializerMethodField()
    is_access_limited = SerializerMethodField()
>>> offer_details.context
Out[23]: 
{}
>>> offer_details._context
Out[24]: 
{}
# ! surprisingly context is available inside fields... but not in the parent
>>> offer_details.fields['is_access_limited'].context
Out[26]: 
{'publisher': <User: John ZusPJRryIzYZ>}

# context is available in all of them!
>>> [x.field_name for x in offer_details.fields.values() if not x.context]
Out[27]: 
[]

@tomchristie sorry to bother, but maybe you'll have any ideas how's that possible?

In a meantime I'm forced to use this hack to solve a problem:

class Serializer(serializers.Serializer):

    def __init__(self, *args, **kwargs):
        super(Serializer, self).__init__(*args, **kwargs)
        # propagate context to nested complex serializers
        if self.context:
            for field in six.itervalues(self.fields):
                if not field.context:
                    delattr(field, 'context')
                    setattr(field, '_context', self.context)

@pySilver we can't help much without a simple test case. You are using several different serializers which may have some user defined code that breaks the context propagation.

It turns out that it happens due to complex mro, where multiple classes are children of serializers.Serializer. Something like that

class MyMixin(serializers.Serializer):
   # some code within __init__, calling parent too

class MyModelSerializer(serializers.ModelSerializer):
   # some code within __init__, calling parent too

class MyProblematicSerializer(MyModelSerializer, MyMixin)
    # this one will not receive context somehow.
     pass

anyway, thats I think might be considered as a local issue..

Thanks for the feedback 👍

@xordoquy @tomchristie actually problem got really strange :) whenever you access self.context in overriden serializer problem with missing context appears. I've tried to do that in fields property with no luck. Here is a testcase:

def test_serializer_context(self):
        class MyBaseSerializer(serializers.Serializer):
            def __init__(self, *args, **kwargs):
                super(MyBaseSerializer, self).__init__(*args, **kwargs)
                x = 0
                if self.context.get('x'):
                    x = self.context['x']

        class SubSerializer(MyBaseSerializer):
            char = serializers.CharField()
            integer = serializers.IntegerField()

        class ParentSerializer(MyBaseSerializer):
            with_context = serializers.CharField()
            without_context = SubSerializer()

        serializer = ParentSerializer(data={}, context={'what': 42})
        assert serializer.context == {'what': 42}
        assert serializer.fields['with_context'].context == {'what': 42}
        assert serializer.fields['without_context'].context == {'what': 42}

However this one would pass. imho it should be noted in sources that accessing context in overriden methods might be problematic...

def test_serializer_context(self):
        class MyBaseSerializer(serializers.Serializer):
            @property
            def fields(self):
                fields = super(MyBaseSerializer, self).fields
                x = 0
                if self.root.context.get('x'):
                    x = 1
                return fields


        class SubSerializer(MyBaseSerializer):
            char = serializers.CharField()
            integer = serializers.IntegerField()

        class ParentSerializer(MyBaseSerializer):
            with_context = serializers.CharField()
            without_context = SubSerializer()

        serializer = ParentSerializer(data={}, context={'what': 42})
        assert serializer.context == {'what': 42}
        assert serializer.fields['with_context'].context == {'what': 42}
        assert serializer.fields['without_context'].context == {'what': 42}

To be clear – what I'm doing here is changing behavior of fields based on context.

UPD: even last code is not reliable in some cases. turns out that touching context is dangerous.

It's unfortunate that __init__ does not have access to the context. If one wants to dynamically create fields in the serializer based on something in the context, where would you do that?

It doesn't seem to work in bind as was suggested above. For example self.fields['asdf'] = serializers.CharField(). I assume the fields have already been evaluated when bind is called.

Doing it in the fields property doesn't seem great either, since that thing is called a lot, and due to the lazy evaluation it has a caching mechanism in self._fields that would have to be duplicated in the concrete serializer. It would be better to be able to modify the fields once, before they are cached internally.

I think this wouldn't be a problem if nesting was done as a field, rather than calling the serializer directly.

Like this:

class ChildSerializer(serializers.Serializer):
    ...

class ParentSerializer(serializers.Serializer):
    child = serializers.NestedField(serializer=ChildSerializer)

Then the ChildSerializer could have access to the context in __init__, since it's not being instantiated directly in the field declaration in ParentSerializer.

If one wants to dynamically create fields in the serializer based on something in the context, where would you do that?

Let's talk about this in the context of a specific example. Come up with a nice simple use-case and we can figure out the best way to tackle it.

I think this wouldn't be a problem if nesting was done as a field, rather than calling the serializer directly.

Try, although I don't think we'll likely go that route. It's a big break in what we currently do, plus it doesn't allow you to set arguments on the child serializer. I think there's a valid case for that style but I don't particularly see us changing at this point in time.

@tomchristie thanks for the reply.

One use case I have right now is to switch between two different sets of fields in a nested child serializer depending on a property of the currently logged in user.

Type A users should never see the fields that are relevant to Type B users, and vice versa.

It looks like this:

class ChildSerializer(serializers.Serializer):
  # relevant for all users
  name = serializers.CharField()

  # only relevant for users of Type A
  phone = serializers.CharField()

  # only relevant for users of Type B
  ssn = serializers.CharField()

But I regularly modify fields in many different ways, in both api's as well as html forms, using drf serializers and/or django forms. Dynamically adding fields, removing fields, modifying attributes, the above example is far from the only way. I just never had to do it in a nested child before, and got completely stumped by this.

@tomchristie my typical usecase is to control whether field is required or not based on context (imagine a serializer for user and admin, where one od them does not need to provide user-id since its passed via context.) by now Im forced to use a solution from my earlier posts in this thread.

I guess one workaround could be to always instantiate the nested ChildSerializer in the __init__ method of the ParentSerializer instead of in the field declaration, then one can forward the context manually.

Like this:

class ChildSerializer(serializers.Serializer):
    ...

class ParentSerializer(serializers.Serializer):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.fields['child'] = ChildSerializer(context=self.context)

That seems to work in an initial test. Not sure if there is any catch with this.

Was going to suggest the same, yup.

Well it did not work for me for some cases. Probably because of multiple inheritance.

On Wed, Oct 12, 2016 at 2:56 PM +0200, "Tom Christie" [email protected] wrote:

Was going to suggest the same, yup.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

I'm having the same problem, when I set default attribute (with CurrentUserDefault() value in my case) to the (nested) serializer field which has subclassed __init__ method which processes self.context attribute, the KeyError appears showing that request isn't found in context (which contains nothing i.e. it doesn't get passed from the parent).
Example code:

class UserSerializer(serializers.ModelSerializer):
# ...
    def __init__(self, *args, **kwargs):
        kwargs.pop('fields', None)
        super().__init__(*args, **kwargs)
        if 'list' in self.context: # Once I remove this, it will work
            self.fields['friend_count'] = serializers.SerializerMethodField()
# ...

class CommentSerializer(serializers.ModelSerializer):
    person = UserSerializer(read_only=True, default=serializers.CurrentUserDefault())
# ...

Error log:

Environment:


Request Method: POST
Request URL: http://127.0.0.1:8000/api/comments/98/

Django Version: 1.9.7
Python Version: 3.4.3
Installed Applications:
['django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.sites',
 'rest_framework',
 'rest_framework.authtoken',
 'generic_relations',
 'decorator_include',
 'allauth',
 'allauth.account',
 'allauth.socialaccount',
 'allauth.socialaccount.providers.facebook',
 'allauth.socialaccount.providers.google',
 'phonenumber_field',
 'bootstrap3',
 'stronghold',
 'captcha',
 'django_settings_export',
 'main']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'stronghold.middleware.LoginRequiredMiddleware']



Traceback:

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/django/core/handlers/base.py" in get_response
  149.                     response = self.process_exception_by_middleware(e, request)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/django/core/handlers/base.py" in get_response
  147.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/django/views/decorators/csrf.py" in wrapped_view
  58.         return view_func(*args, **kwargs)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
  466.             response = self.handle_exception(exc)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/views.py" in dispatch
  463.             response = handler(request, *args, **kwargs)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/generics.py" in post
  246.         return self.create(request, *args, **kwargs)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/mixins.py" in create
  20.         serializer.is_valid(raise_exception=True)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/serializers.py" in is_valid
  213.                 self._validated_data = self.run_validation(self.initial_data)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/serializers.py" in run_validation
  407.         value = self.to_internal_value(data)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/serializers.py" in to_internal_value
  437.                 validated_value = field.run_validation(primitive_value)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/serializers.py" in run_validation
  403.         (is_empty_value, data) = self.validate_empty_values(data)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/fields.py" in validate_empty_values
  453.             return (True, self.get_default())

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/fields.py" in get_default
  437.                 self.default.set_context(self)

File "/home/mikisoft/django-dev/lib/python3.4/site-packages/rest_framework/fields.py" in set_context
  239.         self.user = serializer_field.context['request'].user

Exception Type: KeyError at /api/comments/98/
Exception Value: 'request'

The added field lacks a few things the serializer do through metaclasses. It's up to you to set them.

As I've already said above - once I remove retrieving of context in __init__ method, everything works fine... So field setting certainly isn't the problem, as the context will be passed from the parent in that case.

Right, was on my phone, didn't notice that part.
Then it's normal since nested serializer are instantiated far before the top serializer is.
The purpose of the context is indeed to pass it far after instantiation happened. So it's no wonder it doesn't work in init.

Okay. For the last note, I just want to point out that like the error log shows, it doesn't appear in __init__ method, but in CurrentUserDefault() object (in its set_context method). But sure that it would also appear in __init__ if there was a direct call to the context parameter, i.e. without checking its existence (because the whole context is blank, since it isn't passed over).
Also, I would like to repeat what's concluded above by someone else - in the case when there is no nesting (and when context is used in instantiation method), everything works as expected.

There was a problem in propagating context to children, it should be solved in https://github.com/encode/django-rest-framework/pull/5304 I guess there should've been many issues related to caching None as root of child serializers that should be solved with this PR.

I ran into this issue recently. This is the solution that worked for me:

class MyBaseSerializer(serializers.HyperlinkedModelSerializer):

    def get_fields(self):
        '''
        Override get_fields() method to pass context to other serializers of this base class.

        If the context contains query param "omit_data" as set to true, omit the "data" field
        '''
        fields = super().get_fields()

        # Cause fields with this same base class to inherit self._context
        for field_name in fields:
            if isinstance(fields[field_name], serializers.ListSerializer):
                if isinstance(fields[field_name].child, MyBaseSerializer):
                    fields[field_name].child._context = self._context

            elif isinstance(fields[field_name], MyBaseSerializer):
                fields[field_name]._context = self._context

        # Check for "omit_data" in the query params and remove data field if true
        if 'request' in self._context:
            omit_data = self._context['request'].query_params.get('omit_data', False)

            if omit_data and omit_data.lower() in ['true', '1']:
                fields.pop('data')

        return fields

In the above, I create a serializer base class that overrides get_fields() and passes self._context to any child serializer that has the same base class. For ListSerializers, I attach the context to the child of it.

Then, I check for a query param "omit_data" and remove the "data" field if it's requested.

I hope this is helpful for anybody still looking for answers for this.

I know this is an old thread, but I just wanted to share that I use a combination of __init__ and bind on my custom mixin to filter the queryset based on the request (in the context). This mixin assumes a get_objects_for_user method exists on the queryset, or uses guardian.shortcuts.get_objects_for_user otherwise. It works very nice for nested serializers, also when POSTing or PATCHing.