We're website developers and long-time users of wpDataTables, but this one has us stumped.
We're trying to create a table from WooCommerce Orders, Users and UsersMeta. We understand that for complex queries, it's best to create a view, so we did. It's called "Orders_View" and the "AS" clause is in the attached .sql file.
There are about 1200 rows in the view currently, but it seems we must have crossed the magic threshold in total size where wpDataTables turns on "Enable server-side processing". We can't seem to turn it off even though we're not at the 3000-4000 rows that your Server-side processing (Lazy Loading) in wpDataTables article mentions. So our first question is what is this limit really based on and is there a way we can increase it if we want to? We LOVE the way browser-based JavaScript data filtering work and would like to continue to use it if we can.
Secondly, we find that filtering with the "Server-side processing" enabled is not working. For example, "Jane" is clearly in the table, but when we search for her, we get 0 rows returned. Per your article, we have assured that our query has no "LIMIT" or "ORDER BY" clauses. So our second question is any idea what the server-side filtering problem is?
In the WordPress debug log, we see:
[18-Apr-2019 20:14:34 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Orders_View ``.`First Name` LIKE '%Jane%' ORDER BY `Order Change Date/Time` DESC' at line 1 for query SELECT SQL_CALC_FOUND_ROWS * FROM `Orders_View` WHERE `` Orders_View ``.`First Name` LIKE '%Jane%' ORDER BY `Order Change Date/Time` DESC LIMIT 50 made by do_action('wp_ajax_get_wdtable'), WP_Hook->do_action, WP_Hook->apply_filters, wdtGetAjaxData, WPDataTable->queryBasedConstruct
So I'm not sure if it's a performance problem, query complexity problem, syntax, problem or what. We'd just like this to work one way or another.
Thanks in advance for your help and advice. We would have to get you credentials if you actually want to access our page or table.
How many columns do you have in the table? There's no exact number, but server-side depends on rows * columns value. So, if you have a lot of columns and were do multiply that number with the number of rows, I believe it would explain why server-side is being automatically turned on. I'll check that with my developers, though.
Now, looking at the SQL file you sent, this is how the view is created, correct? I mean - the query within wpDataTables is something like this:
SELECT * FROM orders_view
Correct?
It shouldn't give you the problem with the filter if that is true. If, however the query in our plugin looks like this:
SELECT u.id as 'User ID',
um1.meta_value as 'Last Name',
um2.meta_value as 'First Name',
um_stone_harbor_street.meta_value AS 'Stone Harbor Street Address',
um_stone_harbor_unit.meta_value AS 'Unit #',
u.user_email AS Email,
um_billing_address_1.meta_value AS 'Billing Address 1',
um_billing_address_2.meta_value AS 'Billing Address 2',
um_billing_city.meta_value AS 'Billing City',
um_billing_state.meta_value AS 'Billing State',
um_billing_postcode.meta_value AS 'Billing Zip Code',
um_billing_country.meta_value AS 'Billing Country',
um_billing_phone.meta_value AS 'Phone',
p.ID as 'Order#',
p.post_status as 'Order Status',
DATE_FORMAT(p.post_date, '%c/%e/%Y %H:%i') AS 'Order Create Date/Time',
DATE_FORMAT(p.post_modified, '%c/%e/%Y %H:%i') AS 'Order Change Date/Time',
oi.order_item_name AS 'Product Description',
pm_product_price.meta_value AS 'Cost',
oim_quantity.meta_value AS 'Quantity',
pm_order_total.meta_value AS 'Order Total',
pm_payment_method.meta_value AS 'Payment Method',
pm_transaction_id.meta_value AS 'Transaction#',
pm_transaction_fee.meta_value AS 'Transaction Fee',
DATE_FORMAT(FROM_UNIXTIME(um_expire_user_date.meta_value), '%Y') AS 'Expire Date'
FROM wp_posts as p
LEFT JOIN wp_postmeta pm on pm.post_id = p.ID
LEFT JOIN wp_users as u on u.ID = pm.meta_value
LEFT JOIN wp_usermeta as um1 on um1.user_id = u.ID
LEFT JOIN wp_usermeta as um2 on um2.user_id = u.ID
LEFT JOIN wp_usermeta AS um_stone_harbor_street ON (u.ID=um_stone_harbor_street.user_id AND um_stone_harbor_street.meta_key='stone_harbor_street')
LEFT JOIN wp_usermeta AS um_stone_harbor_unit ON (u.ID=um_stone_harbor_unit.user_id AND um_stone_harbor_unit.meta_key='stone_harbor_unit')
LEFT JOIN wp_usermeta AS um_billing_address_1 ON (u.ID=um_billing_address_1.user_id AND um_billing_address_1.meta_key='billing_address_1')
LEFT JOIN wp_usermeta AS um_billing_address_2 ON (u.ID=um_billing_address_2.user_id AND um_billing_address_2.meta_key='billing_address_2')
LEFT JOIN wp_usermeta AS um_billing_city ON (u.ID=um_billing_city.user_id AND um_billing_city.meta_key='billing_city')
LEFT JOIN wp_usermeta AS um_billing_state ON (u.ID=um_billing_state.user_id AND um_billing_state.meta_key='billing_state')
LEFT JOIN wp_usermeta AS um_billing_postcode ON (u.ID=um_billing_postcode.user_id AND um_billing_postcode.meta_key='billing_postcode')
LEFT JOIN wp_usermeta AS um_billing_country ON (u.ID=um_billing_country.user_id AND um_billing_country.meta_key='billing_country')
LEFT JOIN wp_usermeta AS um_billing_phone ON (u.ID=um_billing_phone.user_id AND um_billing_phone.meta_key='billing_phone')
LEFT JOIN wp_usermeta AS um_expire_user_date ON (u.ID=um_expire_user_date.user_id AND um_expire_user_date.meta_key='_expire_user_date')
LEFT JOIN wp_postmeta AS pm_order_total on (p.ID=pm_order_total.post_id AND pm_order_total.meta_key='_order_total')
LEFT JOIN wp_woocommerce_order_items AS oi on (p.ID=oi.order_id)
LEFT JOIN wp_woocommerce_order_itemmeta AS oim_quantity on (oi.order_item_id=oim_quantity.order_item_id AND oim_quantity.meta_key='_qty')
LEFT JOIN wp_woocommerce_order_itemmeta AS oim_product_id ON (oi.order_item_id=oim_product_id.order_item_id AND oim_product_id.meta_key='_product_id')
LEFT JOIN wp_postmeta AS pm_product_price ON (oim_product_id.meta_value = pm_product_price.post_id AND pm_product_price.meta_key = '_price' )
LEFT JOIN wp_postmeta AS pm_payment_method ON (p.ID = pm_payment_method.post_id AND pm_payment_method.meta_key = '_payment_method' )
LEFT JOIN wp_postmeta AS pm_transaction_id ON (p.ID = pm_transaction_id.post_id AND pm_transaction_id.meta_key = '_transaction_id' )
LEFT JOIN wp_postmeta AS pm_transaction_fee ON (p.ID = pm_transaction_fee.post_id AND pm_transaction_fee.meta_key = 'PayPal Transaction Fee' )
WHERE p.post_type = 'shop_order'
AND pm.meta_key = '_customer_user'
AND um1.meta_key = 'last_name'
AND um2.meta_key = 'first_name'
Then, I believe the plugin is not able to successfully read the query. It probably has problems parsing the query.
Basically, if you were to create a view from the query above, and pull it in wpDataTables in a single line SELECT * FROM view - it should be working fine.
Yes, "select * from Orders_View". Short and sweet. :-)
The view has 2390 rows and 25 columns. No cells are particularly big. Usual strings (Name, Address, City, etc.) and integers.
I would assume that the limit is not truly rows or columns but on the amount of data. It would be great to find out from the developer if this threshold can be tweaked (or "hacked") if we're willing to accept the consequences.
Beyond that, finding out why the server-based search doesn't work would be our second course of action.
The server-side is being turned on automatically after 2000 rows after all. So, with this in mind - there is a way to change this to 3000 or 4000, but it will cause the table to load slower.
If you'd still like this to be implemented, please let me know, and I'll try to make it work for you during the weekend. Also, it is worth mentioning that once the plugin updates the process will have to be repeated because an update will overwrite the workaround.
Great. We understand the implications of changing a plugin file vs. new releases. If you could just tell us which file(s) and line numbers define the limit, we could probably take it from there.
Sure, that info doesn't contain any personal data, so I can share it with you again.
You would need to change the following files:
../wp-content/plugins/wpdatatables/wpdatatables.php - Add this line of code:
if (!defined('WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD')) {
define('WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD', 2000); // Number of rows that triggers manditory server-side processing
}
Then, in ../wp-content/plugins/wpdatatables/source/class.wpdatatable.php change line 2092 from:
if (count($res_dataRows) > 2000) {
to:
if (count($res_dataRows) > WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD) {
In ../wp-content/plugins/wpdatatables/source/class.wdtconfigcontroller.php change line 53 from:
if (count($wpDataTable->getDataRows()) > 2000) {
to:
if (count($wpDataTable->getDataRows()) > WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD) {
And in ../wp-content/plugins/wpdatatables/templates/admin/table-settings/table_settings_block.inc.php change line 167 from:
title="<?php _e('If it is turned on, all sorting, filtering, pagination and other data interaction will be done by MySQL server. This feature is recommended if you have more than 2000-3000 rows. Mandatory for editable tables.', 'wpdatatables'); ?>">
to:
title="<?php _e('If it is turned on, all sorting, filtering, pagination and other data interaction will be done by MySQL server. This feature is recommended if you have more than '.WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD.' rows. Mandatory for editable tables.', 'wpdatatables'); ?>">
So, then a user can add a block like this to wp-config.php to increase the limit (at their own risk):
// Override wpDataTables default threshold of "2000" rows for forced server-side processing
define('WDT_FORCED_SERVER_SIDE_PROCESSING_ROW_COUNT_THRESHOLD', 100000);
LarryDaniele shared this with us, and made these exact changes and it is working well for them.
We're website developers and long-time users of wpDataTables, but this one has us stumped.
We're trying to create a table from WooCommerce Orders, Users and UsersMeta. We understand that for complex queries, it's best to create a view, so we did. It's called "Orders_View" and the "AS" clause is in the attached .sql file.
There are about 1200 rows in the view currently, but it seems we must have crossed the magic threshold in total size where wpDataTables turns on "Enable server-side processing". We can't seem to turn it off even though we're not at the 3000-4000 rows that your Server-side processing (Lazy Loading) in wpDataTables article mentions. So our first question is what is this limit really based on and is there a way we can increase it if we want to? We LOVE the way browser-based JavaScript data filtering work and would like to continue to use it if we can.
Secondly, we find that filtering with the "Server-side processing" enabled is not working. For example, "Jane" is clearly in the table, but when we search for her, we get 0 rows returned. Per your article, we have assured that our query has no "LIMIT" or "ORDER BY" clauses. So our second question is any idea what the server-side filtering problem is?
In the WordPress debug log, we see:
[18-Apr-2019 20:14:34 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Orders_View ``.`First Name` LIKE '%Jane%' ORDER BY `Order Change Date/Time` DESC' at line 1 for query SELECT SQL_CALC_FOUND_ROWS * FROM `Orders_View` WHERE `` Orders_View ``.`First Name` LIKE '%Jane%' ORDER BY `Order Change Date/Time` DESC LIMIT 50 made by do_action('wp_ajax_get_wdtable'), WP_Hook->do_action, WP_Hook->apply_filters, wdtGetAjaxData, WPDataTable->queryBasedConstruct
So I'm not sure if it's a performance problem, query complexity problem, syntax, problem or what. We'd just like this to work one way or another.
Thanks in advance for your help and advice. We would have to get you credentials if you actually want to access our page or table.
Hello LarryDaniele.
Thank you for being our long-term customer.
How many columns do you have in the table? There's no exact number, but server-side depends on rows * columns value. So, if you have a lot of columns and were do multiply that number with the number of rows, I believe it would explain why server-side is being automatically turned on. I'll check that with my developers, though.
Now, looking at the SQL file you sent, this is how the view is created, correct? I mean - the query within wpDataTables is something like this:
Correct?
It shouldn't give you the problem with the filter if that is true. If, however the query in our plugin looks like this:
Then, I believe the plugin is not able to successfully read the query. It probably has problems parsing the query.
Basically, if you were to create a view from the query above, and pull it in wpDataTables in a single line SELECT * FROM view - it should be working fine.
Best regards.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Hi Aleksandar,
Thanks for your help with this.
Yes, "select * from Orders_View". Short and sweet. :-)
The view has 2390 rows and 25 columns. No cells are particularly big. Usual strings (Name, Address, City, etc.) and integers.
I would assume that the limit is not truly rows or columns but on the amount of data. It would be great to find out from the developer if this threshold can be tweaked (or "hacked") if we're willing to accept the consequences.
Beyond that, finding out why the server-based search doesn't work would be our second course of action.
Larry
Hey LarryDaniele.
The server-side is being turned on automatically after 2000 rows after all. So, with this in mind - there is a way to change this to 3000 or 4000, but it will cause the table to load slower.
If you'd still like this to be implemented, please let me know, and I'll try to make it work for you during the weekend. Also, it is worth mentioning that once the plugin updates the process will have to be repeated because an update will overwrite the workaround.
I'll wait for your reply.
Best regards.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Hi Aleksandar,
Great. We understand the implications of changing a plugin file vs. new releases. If you could just tell us which file(s) and line numbers define the limit, we could probably take it from there.
Thanks,
Larry
And it would be helpful to upgrade your article from "3000-4000" to "2000" if that's the case. And perhaps document any work-around. Thanks!
Larry
Thanks Alexsandar. That was very helpful.
You are welcome LarryDaniele. Glad I could help.
If you have any more issues or questions feel free to open a new ticket, we will gladly help.
We'd greatly appreciate it if you could take a minute and leave a Review on CodeCanyon on this link. Thanks!
Best regards.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Thank you very much Larry!
I will forward this to our development team immediately, so they can take it into consideration.
Best regards.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Hi. I have this same exact problem, but i cannot sea what did you modify to solve it. Can you help me?
Hi Jorge.
Sure, that info doesn't contain any personal data, so I can share it with you again.
You would need to change the following files:
../wp-content/plugins/wpdatatables/wpdatatables.php - Add this line of code:
Then, in ../wp-content/plugins/wpdatatables/source/class.wpdatatable.php change line 2092 from:
to:
In ../wp-content/plugins/wpdatatables/source/class.wdtconfigcontroller.php change line 53 from:
to:
And in ../wp-content/plugins/wpdatatables/templates/admin/table-settings/table_settings_block.inc.php change line 167 from:
to:
So, then a user can add a block like this to wp-config.php to increase the limit (at their own risk):
LarryDaniele shared this with us, and made these exact changes and it is working well for them.
I hope this helps.
Best regards.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables