Laravel-datatables: Data loading is very slow for big number of rows.

Created on 8 Mar 2017 · 14Comments · Source: yajra/laravel-datatables

Summary of problem or feature request

I tried to implement Datatable with Laravel but I have problem when my table has big number of rows because Datatable get ALL in one request and then pagination is on JS side. Can I use Laravel pagination or can I return paginated rows with paginate() method from Eloquent on ajax method ? Because this is big problem for my project. Data loading is very slow. When I search some word processing is also very slow and after some operation like sort, search etc. Page freeze and nothing happen.
I hope that someone has same problem like this.

Code snippet of problem

                serverSide: true,
                processing: true,
                render: true,
                columns: [
                    {data: 'code'},
                    {data: 'name', "defaultContent": "<i>Not set</i>"},
                    {data: 'native_name',  "defaultContent": "<i>Not set</i>"},
                    {data: 'action'}
                ],
                ajax: '{!! route('dt-languages') !!}'

System details

Operating System Linux
PHP Version 7.0.4
Laravel Version 5.4
Laravel-Datatables Version 7.2

Source

hilioski

Most helpful comment

You need to remove get to use the query builder.

return $this->languages->select('languages.*');

yajra on 9 Mar 2017

👍5 🎉3 ❤1

All 14 comments

You might be using collection instead of query? Can you paste your server-side code? Thanks!

yajra on 9 Mar 2017

I use repository pattern and in my LanguageRepository.php I have:
return $this->languages->select('languages.*')->get();

hilioski on 9 Mar 2017

You need to remove get to use the query builder.

return $this->languages->select('languages.*');

yajra on 9 Mar 2017

👍5 🎉3 ❤1

Okey. Now looks better. I will test it again and will write my results.
But in the meantime one more problem is there. When I try to Search some word, I get this error:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'languages.action' in 'where clause' (SQL: select count(*) as aggregate from (select '1' as `row_count` from `languages` where (LOWER(`languages`.`name`) LIKE %m% count_row_table)

hilioski on 9 Mar 2017

Set action as not searchable/orderable since it's added column.

yajra on 9 Mar 2017

Thank you very much! @yajra
When I finish with testing I will send results to you

hilioski on 9 Mar 2017

My testing results:

First query:
select count(*) as aggregate from (select '1' as row_count from languages where languages.deleted_at is null and languages.deleted_at is null) count_row_table -> 3.64ms

Second query:
select languages.* from languages where languages.deleted_at is null order by code asc limit 10 offset 0 -> 1.05ms

Thanks @yajra once again for help.

hilioski on 9 Mar 2017

👍1

Glad I was able to help! Thanks!

yajra on 9 Mar 2017

👍1

I am suffering from the problem. Slow rendering. My server side query.

 $products = DB::connection("mysql2")->table('products')->
        join('products_description', 'products.products_id', '=', 'products_description.products_id')
            ->select(['products.products_id', 'products_description.products_name', 'products.products_isbn13', 'products.products_distributor', 'products.products_price', 'products.products_status']);
 return Datatables::of($products)->make(true);

help me. Should I write my own script to paginate? what should I do?
I have records of more than 4,000,000 products.
@yajra very appreciating if you help me.

hsali on 8 Aug 2017

👍7

I got this one too, may be the datatable request not chaining the pagination param to the query?

pnghai on 5 Oct 2017

👎1 👍1

@pnghai how I can check it?

hsali on 18 Oct 2017

@hsali I'm sorry. False alarm. Turned out that my logic involved a remote image fetching, that links to a dead-end server and force each record fetch wrong 404 connection, that take up number of record * 60 (seconds before timeout). I suspended that logic and things work very well now.

Thank you very much
(p/s: the pnghai nick is another github account of myself)