Okay
  Public Ticket #3820213
Slow rendering of table
Open

Comments

  • Mateusz Kopacz started the conversation

    I have a DataTable that is using an SQL view. The database contains a large amount of data, but the SQL view is running fast (< 0.3 seconds).

    The issue is that the wpDataTable is running very slowly. I checked the Query Monitor, and the problem seems to be related to the filters. They are attempting to read values during the table's loading process. Please see the attached screenshot for further details.

    Best regards,
    Mateusz

    Attached files:  filteringIssues.png

  • Mateusz Kopacz replied

    Can I have some update here? It's super urgent

  •  1,876
    Miloš replied

    Hello,

    My sincerest apology, there was an error on this ticket from my end - I wrote a reply, but it somehow did not get sent properly.

    This will not happen again, thank you very much for your patience.

     

    1. First, we will advise in general about using SQL Queries via our Plugin, and how you can try to simplify them for better performance.
      It also depends if you use our server-side processing or not and the size of the table.

      if the table has less than 2 thousand rows, you can try to disable the server-side option and see if it loads quicker.

      This can help if you have a complex Query.

      - But, since you said it is inside a VIEW, how are you calling that view in our Table?

      Try to simplify our Table's Query as much as possible, so just like : SELECT * FROM VIEWName.

      Here are more details which could be helpful on this sutject, in regards to how our server-side processing works for complex Queries :


       

      Our SQL Feature is not the same as a Database Management Tool, so for example - a specific Custom Query might work in your SQL Database directly but not in our SQL Feature of the Plugin, because we run everything through our PHP & SQL Parser, which has certain limitations.

      We will do our best to advise.

      On this documentation, there are more details which explain how our server-side processing works :

      https://wpdatatables.com/documentation/table-features/server-side-processing/.

      Basically, when the server-side option in a Table is enabled, the wpDataTables will give the search results through the Query;

      5716080920.png

      So, our Plugin sends the Query to the database.

      If that Query is formatted as

       "SELECT ...

       FROM ... 

      WHERE... "

      but after the "FROM" if it has any complex Query, there can be errors;

      as we mentioned, our SQL Query feature does not work in the same way as a Database Management Tool ( such as PHPMyAdmin and similar),

      and is not meant to be used as one;

      it is based on an SQL Parser which has limitations;

      so it can happen that a specific complex Query might work in PhpMyAdmin but struggles to work through our server-side processing ( and the SQL Parser);

      So when the Server-Side is enabled, our plugin sends a more complex SQL Query which in this case is too complicated for our Parser to handle,

      instead of when the server-side is disabled, it just simply filters the data just by the values already seen in the column.

      I hope that this helps to clarify everything, did my best to pass our developer Team's explanations.

      So to solve this particular issue, you would need to either try and somehow simplify your SQL Query in order for our server-side processing to work;

      or if you can make it work without server-side processing, if your data/number of rows of  the table does not become too large, let's say above 4, 5, 6 thousand rows,  and if the hosting server performs well,

      you can 'get away' with disabling server-side on the table and see if that helps.

      Here is how you can 'raise' the limit of rows for "server-side processing" toggle.

       If your SQL query based tables are not bigger than 2.000 rows, 

      you can disable server-side on SQL tables, and it will work like it does for Excel tables. ( loads all rows regardless of pagination)

      If you need to increase the row count while still having the "toggle" to disable server-side,

      Please go to ../wp-content/plugins/wpdatatables/source/class.wpdatatable.php and around line 2176 you'll see this:

      if (count($res_dataRows) > 2000) {

      You can change that number to a value bigger than the number of rows in your table.

      Same should be applied in ../wp-content/plugins/wpdatatables/source/class.wdtconfigcontroller.php on lines 53:

      if (count($wpDataTable->getDataRows()) > 2000) {
      

      And line 100:

      if (count($wpDataTable->getDataRows()) > 2000) {
      

      That will increase the server-side automatic limit.

      Let us know if you manage to simplify the Query for our Parser, or if disabling Server-Side processing helps;

      If none of that helps,

      you can try preparing a MySQL view (which will return the data that you need, call it e.g. “view1” and then build a wpDataTables based on a simple query like "SELECT * FROM view1″.

       If you need help with that, you can see our video, where we show an example of using View in our plugin.

      I hope one of these methods might solve the issue.




    2. Now, about the loading speed in general :

      The loading speed depends on a lot of factors - your hosting plan, server's performance, amount of other data on the page, internet speed, etc. 

    For example, you can take a look at the table on our documentation, which contains around 2.600.000 rows, and see how it loads. The more data the table has, the more chance that it will impact the loading time.

    Please note that Table is an SQL Query Based Table, it uses our Server-Side Processing feature, which dramatically improves loading speeds by using AJAX to only load one Table page at a time,

    compared to non-server-side Tables which always load all rows on the HTML of the Page regardless of Table Pagination.

    -

     

    If you use non-server side Tables, like linked from Google Spreadsheet/CSV/Excel , we advise trying our cache and Auto Update Cache feature.

    With our caching option, the load time is significantly lowered, but we still recommend keeping the data in these files down to around 5.000 rows.

    If any of your Table linked from source file goes above that size - then the best is to switch to SQL based Tables, to import the data from CSV/Google Sheet or Excel.

     

     If you import the file, the plugin reads the source file and creates a manual table

    Once the table is created, it is no longer linked to the source file, so changing data in the table will not show up in Excel or Google Spreadsheet, and vice-versa. 

    To modify this table, you can either switch to the Excel-like view

    and then select and copy the range of cells you want to paste from the source file, and paste it in the Excel-like view, 

    or you can import a new/edited CSV to update the table, as well, in the following ways :

    -

     

    You can also check, in this table's filter settings for columns, how many unique filtering values there are.

    If you have, for example - a string column that you set as a selectbox/multiselectbox, 

    and if it has more than 50 possible filtering values,

    you can go into that column setting, and check what is set for "Number of possible values to load" – 

    Here you can define how many possible values per page will be loaded in selectbox filter and editor inputs. It is recommended not to be set for All if you have more than 50 possible values for this column.

    2095790605.png
    1309831925.png

    So, check if that might be the case, try to lower the value to 50 and see if it helps the speed.

    -

     

    Also, it is possible for other plugins to affect the loading speed.

    We had a couple of cases with users who have an "optimization plugin" with specific settings, 

    such as  'SiteGround Optimizer', for example.

    These plugins have certain settings that can negatively affect the performance of our tables.

    If there is any JS minification going on,  there can be other settings, I can't say exactly,

    but something to be aware of.

    If you wish to do troubleshoot testing to check if an optimization/caching plugin might be slowing down tables,

    you can try to disable all plugins and just leave wpDataTables active - then clear all cache and reload the page, see if it loads faster.

     

    Then, if any optimization plugin Is used, try to revert all its settings to default, and if you use server-side tables ( such as SQL or Manual tables) ,  then exclude the pages where you have tables from caching.

     

    -

    Another thing to consider is how many 'Default rows per page' are set in the Display settings.

    2392875160.png

    Try setting this as low as possible and test if that improves performance for that table.

    I recommend it is best to use our new 'Custom rows per page', so if you allow front-end users to be able to use the dropdown for Rows per page on front-end, this will limit their options, to avoid loading 'All rows' and if you wish to get as much performance as possible, you can even remove the 100 and 50 options, or even 25 if that helps. 

    8251761984.png

     

    -

    Check if you have the "auto-refresh" enabled on the Tables.

    7850453827.png

     

    If you have a large number of tables and enabled this at a high frequency, such as every 5 or 10 seconds, then the Hosting Server will often make Admin-Ajax calls from the Hosting Server to the SQL Database to pull the data, which can increase the server's load.

    You can try disabling it and see if that helps.

    -

    Lastly, if you use an external Database connection ( not the default WordPress database), in some cases it can help the performance if you can move those SQL tables to pull them directly from the same DB where WordPress is on the Hosting server, so you can try that as well.

    -

    That is about all the advice i can give about this, but of course, let us know if you have additional questions. Thank you.

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables

  • Mateusz Kopacz replied

    Hi,

    Thank you for your answers. However, I believe there is a misunderstanding. I need server-side processing because the data set contains approximately 50,000 rows.

    This is the query I use:


    —but the issue remains.

    I discovered that one problem was with the selectbox/multiselectbox filters. I changed their settings in the "Possible values for column" field to use predefined values, which solved that particular issue. 

    However, I am now facing another problem. Although the page loads relatively quickly (about 2.5–3.5 seconds), and the query monitor shows that the data query connected to the datatable is executed, I see an additional screen for about 20 seconds (see attached file). It seems that the table is rendered for approximately 20 seconds, or something unusual is happening that I don’t understand.

    Could you please help me figure out what might be causing this delay?

    Thank you for your assistance.

    BTW I cannot paste the query - I cannot save the comment when I'm pasting query

  • Mateusz Kopacz replied

    screenshot

    Attached files:  renderingIssuepng.png

  • Mateusz Kopacz replied

    The reply I tried to save

    Attached files:  commentissue.png

  • Mateusz Kopacz replied

    Sorry to push but do you have any update here?

  • Mateusz Kopacz replied

    Could you answer my questions? 

  •  1,876
    Miloš replied

    Hi Mateusz,
    My sincere apologies for all the waiting time. Thank you for your patience.

    In regards to these issues :

    1. Since you have so much data/rows, as you mentioned at one point, you noticed that the dropdown filter was loading too slow.
    Yes, the solution is to limit how many possible values can be loaded for the dropdown;
    we will share more details on that later on, in the 'slow loading' section.

    First, I wish to advise on that subject, in a situation when you have 'too many unique values possible for a column which has a dropdown filter.
    If the issue is coming from too many entries in the filter.

     

    All of them are being rendered in HTML of the page and it is passing through a piece of the code in WordPress where it can not handle that much data.

    - When the Data is set to "All" (and you have , for example, over 12.000 unique values for the filter), the content breaks.
    4308732317.png

    The solution is to add this to the funcions.php file of your theme, or child theme 

    ( we recommend using a child theme because if you update a theme, it can overwrite this):

    add_filter( 'option_use_smilies', '__return_false' );


    Or, you can limit the number of Possible values for column as low as possible, to improve the loading speed, it is not recommended to set it more than 50 for larger tables, as the tooltip says :
    8291470077.png
    2. Here are more details, which you can check and possible workaround solutions, to improve slow load issues :

    The loading speed depends on a lot of factors - your hosting plan, server's performance, amount of other data on the page, internet speed, etc. 

    For example, you can take a look at the table on our documentation, which contains around 2.600.000 rows, and see how it loads. The more data the table has, the more chance that it will impact the loading time.

    Please note that Table is an SQL Query Based Table, it uses our Server-Side Processing feature, which dramatically improves loading speeds by using AJAX to only load one Table page at a time,

    compared to non-server-side Tables which always load all rows on the HTML of the Page regardless of Table Pagination.

     

    If you use non-server side Tables, like linked from Google Spreadsheet/CSV/Excel , we advise trying our cache and Auto Update Cache feature.

    With our caching option, the load time is significantly lowered, but we still recommend keeping the data in these files down to around 5.000 rows.

    If any of your Table linked from source file goes above that size - then the best is to switch to SQL based Tables, to import the data from CSV/Google Sheet or Excel.

     

     If you import the file, the plugin reads the source file and creates a manual table

    Once the table is created, it is no longer linked to the source file, so changing data in the table will not show up in Excel or Google Spreadsheet, and vice-versa. 

    To modify this table, you can either switch to the Excel-like view

    and then select and copy the range of cells you want to paste from the source file, and paste it in the Excel-like view, 

    or you can import a new/edited CSV to update the table, as well, in the following ways :

     

     

    You can also check, in this table's filter settings for columns, how many unique filtering values there are.

    If you have, for example - a string column that you set as a selectbox/multiselectbox, 

    and if it has more than 50 possible filtering values,

    you can go into that column setting, and check what is set for "Number of possible values to load" – 

    Here you can define how many possible values per page will be loaded in selectbox filter and editor inputs. It is recommended not to be set for All if you have more than 50 possible values for this column.

    2095790605.png
    1309831925.png

    So, check if that might be the case, try to lower the value to 50 and see if it helps the speed.

     

    Also, it is possible for other plugins to affect the loading speed.

    We had a couple of cases with users who have an "optimization plugin" with specific settings, 

    such as  'SiteGround Optimizer', for example.

    These plugins have certain settings that can negatively affect the performance of our tables.

    If there is any JS minification going on,  there can be other settings, I can't say exactly,

    but something to be aware of.

     

    If you wish to do troubleshoot testing to check if an optimization/caching plugin might be slowing down tables,

    you can try to disable all plugins and just leave wpDataTables active - then clear all cache and reload the page, see if it loads faster.

     

    Then, if any optimization plugin Is used, try to revert all its settings to default, and if you use server-side tables ( such as SQL or Manual tables) ,  then exclude the pages where you have tables from caching.

     

     

    Another thing to consider is how many 'Default rows per page' are set in the Display settings.

    2392875160.png

    Try setting this as low as possible and test if that improves performance for that table.

    I recommend it is best to use our new 'Custom rows per page', so if you allow front-end users to be able to use the dropdown for Rows per page on front-end, this will limit their options, to avoid loading 'All rows' and if you wish to get as much performance as possible, you can even remove the 100 and 50 options, or even 25 if that helps. 

    8251761984.png

     

     

    Check if you have the "auto-refresh" enabled on the Tables.

    7850453827.png

     

    If you have a large number of tables and enabled this at a high frequency, such as every 5 or 10 seconds, then the Hosting Server will often make Admin-Ajax calls from the Hosting Server to the SQL Database to pull the data, which can increase the server's load.

    You can try disabling it and see if that helps.

     

    Lastly, if you use an external Database connection ( not the default WordPress database), in some cases it can help the performance if you can move those SQL tables to pull them directly from the same DB where WordPress is on the Hosting server, so you can try that as well.

     

    That is about all the advice I can give about this subject, but of course, let us know if you have additional questions.

     

    3. One more important point - If the table wont' load on the front-end, check can you see an "Invalid JSON response" in the browser console.
    If so, check do you use any active caching on the WP Page with a server-side table.

    We currently have this issue which only happens for server-side processing tables when you have caching active on the page.

     

    Our developers are intensively working to resolve this issue, as soon as possible. I can't say an ETA on it, unfortunately.

     

    You can certainly follow our changeLog page if you'd like ( it is also available in the plugin dashboard), where we state any changes/new features/bug fixes during updates;

     

    and our newsletter, so you're informed about new features, bug fixes, freebies, etc.

     

     

     

    For the time being, the only two possible solutions will be:

     

    1.  To disable caching only for pages where you have the tables, or perhaps disable caching entirely, if possible.

     

    Most caching Plugins have the option to add URL's that will be excluded, so if possible, see if you can add the Pages with tables there, and that should solve it for now.

     
     

    2. Or you can try to completely disable caching on Site.

     
     
     

    Let us know how it goes and if you have any questions.

     

    I hope that helps.

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables

  • Mateusz Kopacz replied

    Hi Milos

    thanks for the answer.  I feel that we don't understand each other :)

    1. Let's start from the begginnig. Originally I page load time was varying between 60-90 seconds. And than I have to wait for another 60-90 seconds to see my table. 

    2. This was resolved by limitig how many possible values can be loaded for the dropdown .

    3. That solved the issue with slow page load time

    4. Now pages loads quickly but I have to wat for 40 seconds to see my table.


    What is identified:

    1) Query to get data to my table is fired on the page_load is called by 

    WPDataTable->queryBasedConstruct
    WPDataTable->fillFromData
    wdtWpDataTableShortcodeHandler
    do_shortcode_tag
    preg_replace_callback
    do_shortcode
    SROB_Submissions_Page->render_page
    SROB_WP_Datatables_Pages->load_page_class
    do_action('toplevel_page_zgloszenia-obozow')

    2. After loading the page there is an ajax-call  to wp-admin/admin-ajax.php?action=get_wdtable&table_id=3
    and this call is taking over 40 secods.


    1) Since there is an ajax to call to get the table - why all of data is get at page load time?

    2) SInce it is an ajax call how can I identify what is taking so much time? 



    Attached files:  Rendering issues.png

  • Mateusz Kopacz replied

    Hi Milos 


    Can I ask for update here?

  • Mateusz Kopacz replied

    Can i ask for update herer?

  •  1,876
    Miloš replied

    Hi Mateusz,

    I sincerely appreciate your patience. As mentioned in one of your older tickets, we’ve been experiencing some issues with our ticketing system. If you ever notice a longer response time, feel free to reach out to my business email (listed in my signature) with your ticket URL or ID, and I’ll do my best to assist you as quickly as possible.

    Regarding your concerns about the slow loading time, I want to reiterate a few key troubleshooting steps that are crucial in diagnosing and resolving this issue:

    Understanding the Admin-Ajax Call

    Since your table is SQL-based and contains a large number of rows, it is expected to use our server-side processing, which relies on AJAX to load one page of data at a time. This is a standard method to handle large datasets efficiently.

    You mentioned that the following AJAX call takes over 40 seconds:
    wp-admin/admin-ajax.php?action=get_wdtable&table_id=3

    To narrow down the cause of the delay, please check the following:

    1. Default Rows Per Page Setting

      • How many rows per page are currently set?
      • Try reducing this number (e.g., set it to 10 or 25) and see if the load time improves.

    2. Total Number of Rows

      • How many total rows does this table have?
      • Tables with more than 2,000+ rows will naturally require optimized queries and efficient server resources.

    3. Caching & Optimization Plugins

      • Do you have any caching or optimization plugins enabled? If so, try disabling them temporarily and check if the load time improves.
      • Some optimization plugins (e.g., SiteGround Optimizer) have settings that interfere with AJAX-based tables. If applicable, try excluding the table page from caching.

    4. Backend vs. Frontend Load Time

      • Does the table load quickly in the WordPress admin panel, but only slow on the front end?
      • If yes, this often points to a conflict with caching, optimization, or other front-end scripts.

    5. Plugin Conflict Testing

      • Temporarily disable all other plugins except wpDataTables and test again.
      • If this resolves the issue, re-enable plugins one by one to identify the conflicting one.

    Additional Considerations

    • SQL Query Complexity: Our demo table (2 million+ rows) loads quickly because it uses a simple query (SELECT * FROM TableName) and an optimized server. If your query is more complex, performance may vary.

    • Filters & Selectbox Settings: If your table has select/multiselect filters, ensure that “Number of possible values to load” is set to a reasonable number (e.g., 50 instead of “All”).

    • Auto-Refresh Feature: If enabled at a high frequency (e.g., every 5-10 seconds), it can put additional load on your server. Try disabling it and see if performance improves.

    • External Database Connections: If the table pulls data from an external database, moving it to the same database as WordPress can improve response time.

    These are the main areas that typically impact table loading speed. Please go through each of these steps and let me know what you find.

    I’d be happy to assist further based on your results.

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables