Mongoose: Virtual nested populate breaks options on parent populate.

Created on 27 May 2018 · 3Comments · Source: Automattic/mongoose

I believe there is a bug using nested populates and options.

My current query is as follows:

...
await Post.find(
          { teacher: req.params.id },
          {},
          {
            sort: { createdAt: -1 },
            limit: 5,
            skip: parseInt(req.query.skip, 10)
          }
        ).populate([
        {
          path: 'comments',
          options: {
            limit: 5,
            sort: { createdAt: -1 }
          },
          populate: { path: 'user', select: '-password' }
        }
      ])
...

When including options on the comments populate (which has another populate nested inside), it returns more than specified limit and sort is not applied. If the nested populate is removed, it works as intended.

Currently, using:

Node 10.2.1
Mongoose 5.1.2
Mongo 3.6.5

Is this a bug or am I missing something?

I believe this might be related: https://github.com/Automattic/mongoose/issues/4321

confirmed-bug

Source

Mikeysax

Most helpful comment

@Mikeysax I created this gist to replicate this as best I could from your example.

It can be cloned locally into a folder called 6528 thusly:

issues: git clone https://gist.github.com/7dee38490d8c637976d8fb0805aafb1e ./6528
Cloning into './6528'...
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), done.
issues: ls !$
ls ./6528
6528.js          models6528.js    populate6528.js
issues:

Output:

6528>: chmod +x populate6528.js
6528>: chmod +x 6528.js
6528>: ./populate6528.js
created 10 users
created 10 teachers
created 100 posts
created 1000 comments
6528>: ./6528.js
{ AssertionError [ERR_ASSERTION]: 10 === 5
    at run (/Users/lineus/dev/Help/mongoose5/issues/6528/6528.js:38:10)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
  generatedMessage: true,
  name: 'AssertionError [ERR_ASSERTION]',
  code: 'ERR_ASSERTION',
  actual: 10,
  expected: 5,
  operator: '===' }
^C
6528>:

I'll be travelling most of the day, but will able to look into it tonight or first thing tomorrow.

lineus on 28 May 2018

👍2

All 3 comments

@Mikeysax I created this gist to replicate this as best I could from your example.

It can be cloned locally into a folder called 6528 thusly:

issues: git clone https://gist.github.com/7dee38490d8c637976d8fb0805aafb1e ./6528
Cloning into './6528'...
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), done.
issues: ls !$
ls ./6528
6528.js          models6528.js    populate6528.js
issues:

Output:

6528>: chmod +x populate6528.js
6528>: chmod +x 6528.js
6528>: ./populate6528.js
created 10 users
created 10 teachers
created 100 posts
created 1000 comments
6528>: ./6528.js
{ AssertionError [ERR_ASSERTION]: 10 === 5
    at run (/Users/lineus/dev/Help/mongoose5/issues/6528/6528.js:38:10)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)
  generatedMessage: true,
  name: 'AssertionError [ERR_ASSERTION]',
  code: 'ERR_ASSERTION',
  actual: 10,
  expected: 5,
  operator: '===' }
^C
6528>:

I'll be travelling most of the day, but will able to look into it tonight or first thing tomorrow.

lineus on 28 May 2018

👍2

Just to be clear, my fix applies the options object to the query as expected, but does not fix the issue in #4321. I will give some thought to that issue separately and submit a PR if I have any ideas worth sharing ( It's a tough nut to crack ).

lineus on 30 May 2018

Yeah limit and populate has been a thorn in our side for a very long time. The general idea is that we'll have to fall back to doing separate queries for each doc if you use populate() with limit and you get multiple docs back.