Okay
  Public Ticket #3562810
Performance Issues with Large Dataset in WPDataTables
Closed

Comments

  • Lucas Petrášek started the conversation

    Dear WPDataTables Support Team,

    I trust this message finds you well. I am currently facing a challenge with my table, which I have imported through an SQL query. The table contains 9.8 million rows, and I'm experiencing significant delays in loading time. Additionally, enabling filters prevents the table from loading at all.

    I am reaching out to seek your guidance on resolving this performance issue. Is there a solution or best practice that can be applied to enhance the loading speed and functionality, especially when working with such a large dataset?

    I would really appreciate your assistance in this matter. Thank you for your time and support.

    Best regards,
    Lucas Petrášek

  •  1,846
    Miloš replied

    Hello,

    The loading speed depends on a lot of factors - your hosting plan, server's performance, amount of other data on the page, internet speed, etc. 

    For example, you can take a look at the table on our documentation, which contains around 2.600.000 rows, and see how it loads. The more data the table has, the more chance that it will impact the loading time.

    Please note that Table is an SQL Query Based Table, it uses our Server-Side Processing feature, which dramatically improves loading speeds by using AJAX to only load one Table page at a time,

    compared to non-server-side Tables which always load all rows on the HTML of the Page regardless of Table Pagination.

    -

    If you use non-server side Tables, like linked from Google Spreadsheet/CSV/Excel , we advise trying our cache and Auto Update Cache feature.

    With our caching option, the load time is significantly lowered, but we still recommend keeping the data in these files down to around 5.000 rows.

    If any of your Table linked from source file goes above that size - then the best is to switch to SQL based Tables, to import the data from CSV/Google Sheet or Excel.

     If you import the file, the plugin reads the source file and creates a manual table

    Once the table is created, it is no longer linked to the source file, so changing data in the table will not show up in Excel or Google Spreadsheet, and vice-versa. 

    To modify this table, you can either switch to the Excel-like view

    and then select and copy the range of cells you want to paste from the source file, and paste it in the Excel-like view, 

    or you can import a new/edited CSV to update the table, as well, in the following ways :

    -

    You can also check, in this table's filter settings for columns, how many unique filtering values there are.

    If you have, for example - a string column that you set as a selectbox/multiselectbox, 

    and if it has more than 50 possible filtering values,

    you can go into that column setting, and check what is set for "Number of possible values to load" – 

    Here you can define how many possible values per page will be loaded in selectbox filter and editor inputs. It is recommended not to be set for All if you have more than 50 possible values for this column.

    2095790605.png
    1309831925.png

    So, check if that might be the case, try to lower the value to 50 and see if it helps the speed.

    -

    Also, it is possible for other plugins to affect the loading speed.

    We had a couple of cases with users who have an "optimization plugin" with specific settings, 

    such as  'SiteGround Optimizer', for example.

    These plugins have certain settings that can negatively affect the performance of our tables.

    If there is any JS minification going on,  there can be other settings, I can't say exactly,

    but something to be aware of.

    If you wish to do troubleshoot testing to check if an optimization/caching plugin might be slowing down tables,

    you can try to disable all plugins and just leave wpDataTables active - then clear all cache and reload the page, see if it loads faster.

    Then, if any optimization plugin Is used, try to revert all its settings to default, and if you use server-side tables ( such as SQL or Manual tables) ,  then exclude the pages where you have tables from caching.


    That is about all the advice i can give about this,

    but of course, let us know if you have additional questions. Thank you

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables

  • Lucas Petrášek replied

    Hi Miloš,

    Thanks alot for your response, I really appreciate it! I will try the solutions you have writen down. I wish you a beautiful day and take care.

    Sincerely,

    Lucas Petrášek

  • Lucas Petrášek replied

    Hi Miloš once again,

    Unfortunately I tried all the solutions you sent me, but it didn't went as I expected. When I enable the filters that I need to have on there, it doesn't load at all - at your app and on the website, BUT when I disable all the filters, and everything (it's basically just a plain database without anything extra), it takes 28 seconds, from all computers that I tried it (with different wifi connections). 

    On PhpMyAdmin, the sql database shows this result (Showing rows 0 - 24 (9762907 total, Query took 0.0005 seconds.) which is the same as a 2 mil database that worked perfectly on my website with your plugin. 

    The question is - Is there a way to show the database through your plugin, or some solution to solve this?

    Thanks a lot in advance, as I said I really appreciate your time and help!


    Warm regards,

    Lucas Petrášek

  •  1,846
    Miloš replied

    Hi Lucas,

    To begin, I want to extend my heartfelt apologies for the delayed reply, 

    which is due to an unexpectedly high volume of inquiries. 

    We truly value your patience during this period.

    -

    Can you tell me is the Table loading in the same speed on Back End and the Front End on a Page?

    If the main issue is on the front-end Load, can you send me the URL of the Page with the Table?


    We should also check the Back-End of the Table and the rest of your Setup.


    Please provide me a temporary WP-admin (administrator) user for your website where this happens, 

    as well as FTP credentials and access to the database ( either link with credentials for PHPmyAdmin, or access to Hosting Panel),

    so we could log in and take a look ‘from the inside’ as that’s the most efficient way to see and resolve the issue. 

    We do not interfere with any data or anything else except for the plugin (in case that’s a production version of the site), and of course, we do not provide login data to third parties. 

    You can write credentials here just check PRIVATE Reply so nobody can see them except us.

    And point me to the Table ID, please.

    Thanks.

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables