I have over 250,000 records that need to be returned and filtered for use in a chart however the table seems to take a long time or just does not load. Are there any recommended alternatives of doing this to generating a query then filtering the table based on a parameter value?
Digging further into the specific error I am seeing this in my logs:
[23-Jun-2020 17:38:03 UTC] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /home2/objecto1/public_html/dataace/wp-content/plugins/wpdatatables/source/class.wpdatatable.php on line 2294
There are wpDataTables that work with millions of entries, so I don't believe that's the case.
You may want to increase the following:
upload_max_file size,
post_max_size,
max_execution_time
You should be able to find these values in your php.ini file or on your hosting panel. If you can't find it yourself please ask your hosting provider to provide it.
Other than that, you may want to look at your server's performance, hosting plan and the amount of other data you have on your pages. SQL query based tables (that don't pull ALL data onto a page) should work normally. Maybe a bit slower than non-server-side (smaller) tables, but work all-in-all.
Thanks Aleksandar. Those settings did not change the behavior. Is there any way I can pass a url paremeter into the where statement of the sql that is executed?
I know I can pass a url parameter to filter a dataset (that's what we are doing now) but that requires pulling all data then filtering. I also know we can use placeholders which helps reduce the number of tables required but shifts us to have multiple pages instead of multiple tables which is really no gain on our part. Is there any way to pass a url parameter into a placeholder?
Although, there is a bug on our end, and I'll have to ask you to bear with me to get it working properly.
The action is called correctly when modifying a table in the backend, but if you have the same table rendered from the front-end the action is not called correctly.
If you change a global variable within this action - works in the backend, in the front-end the default value is used:
function testSetVar1toMNR($MyTable){
global $wdtVar1;
//get Column from Plugin
$lMNR = do_shortcode('[ihc-user field="CUSTOM_FIELD_column"]');
//Found?
if ($lMNR>0) {
$wdtVar1 = $lMNR; //Set VAR1 custom parameter to column
} else {
$wdtVar1 = "9999";
//Default to 9999 --> needed for Admin Context
}
}
add_action('wpdatatables_before_get_table_metadata', 'testSetVar1toMNR');
Unfortunately at the moment I can offer only this Hot-Fix with non desired-side-effects:
public function fillFromData($tableData, $columnData) {
if (empty($tableData->table_type)) {
return;
}
global $wdtVar1, $wdtVar2, $wdtVar3;
//test: Change 31.07.2019 to update placeholders in production mode
//needs default value for admin-context, otherwise breaks table creation
do_action('wpdatatables_before_get_table_metadata');
// Set placeholders
$wdtVar1 = $wdtVar1 === '' ? $tableData->var1 : $wdtVar1;
….SNIP
Here: class.wpdatatable.php
I hope this helps.
After a new update is released, you won't need to add the hot-fix, though.
We've had a new update is released, so you won't need to add the hot-fix.
The first part of the code goes into functions.php of your theme, while the other part of the code goes in ../wp-content/plugins/wpdatatables/source/class.wpdatatable.php. You can look for "public function fillFromData" in that file (should be around line 2722), and overwrite it.
Please save a copy of the original file, just in case something goes wrong, so you can restore it.
To be able to pass variables dynamically through URL, you should try something like this:
And in the browser:
That would return "10" and overwrite the original %VAR1% value.
I have upgraded and entered the following but I'm getting the error that no data is returned in the table. It appears that quotes that are need to select strings are not being added to %VAR1% parameter.
Rendered query: SELECT geo, parent FROM WorldStats WHERE Parent = Wisconsin LIMIT 10 MySQL said: Unknown column 'Wisconsin' in 'where clause'
Please provide me a temporary WP-admin login for your site where this happens, so we could log in and take a look ‘from the inside’ as that’s the most efficient way to see and resolve the issue. We do not interfere with any data or anything else except for the plugin (in case that’s a production version of the site), and of course we do not provide login data to third party.
You can write credentials here just check Private Reply so nobody can see them except us.
Aleksandar is out of office today and I will help you out.
We check your website, and we notice that you only copy and paste code for hook that is the example from other customer needs which is the reason why your tables with placeholder VAR1 is not working. (attachment)
Another this that we found in table with id 88 is that you are using in condition Parent instead of parent. (attachment)
Please note that using hooks and filters requires some programming skills so you can adopt functionality for your needs.
If you want to filter the table with URL parameter you need to change hook example that Aleksandar provide to you.
Here are some advices in comments
function setVar1fromURL($tableID){
global $wdtVar1;
//get URL parametar
$varFromURL = $_GET['wdtVAR_1'];
// Here is the condition that ask is that URL parameter set
// Code like this will be executed for every table with SQL query
// If you need for specific table then you will need to provide additional code
// for example you want to filter only table with id 1 then
// you will replace if (isset($varFromURL)) { with this if (isset($varFromURL && $tableID == 1)) {
if (isset($varFromURL)) {
$wdtVar1 = $varFromURL; //Set VAR1 with parameter from URL
}
// if your condition is not true then this code below will be executed
// reason why your tables is not working because $wdtVar1 is always set to 9999
// if URL parameter wdtVAR_1 is not set
// if you don't need it you can remove it
// This is example for some other customer needs so you need to adopt for your needs
else {
$wdtVar1 = "9999"; //Default to 9999 --> needed for Admin Context
}
}
add_action('wpdatatables_before_get_table_metadata', 'setVar1fromURL');
My need is very simple for now. I have tried both of the below examples in functions.php. It appears that there is no element named table in the $parsedQuery['FROM'][0] in class.wpdatatable.php.
The functions error is likely happening in edit because there is no wdtVAR_1 here and it should instead load the default defined in Placeholders. It's as if isset is not acting as expected.
class.wpdatatable.php on line 1501 $tableName = $parsedQuery['FROM'][0]['table'];
functions.php on line 167 if (isset($varFromURL) && ($tableID == 93 || $tableID == 94 || $tableID == 99)) { if (isset($varFromURL)) {
All additional Functions.php code
function setVar1fromURL($tableID){ global $wdtVar1; //get URL parametar $varFromURL = $_GET['wdtVAR_1']; // Here is the condition that ask is that URL parameter set // Code like this will be executed for every table with SQL query // If you need for specific table then you will need to provide additional code // for example you want to filter only table with id 1 then // you will replace if (isset($varFromURL)) { with this if (isset($varFromURL && $tableID == 1)) { //if (isset($varFromURL) && ($tableID == 93 || $tableID == 94 || $tableID == 99)) { if (isset($varFromURL)) { $wdtVar1 = $varFromURL; //Set VAR1 with parameter from URL } // if your condition is not true then this code below will be executed // reason why your tables is not working because $wdtVar1 is always set to 9999 // if URL parameter wdtVAR_1 is not set // if you don't need it you can remove it // This is example for some other customer needs so you need to adopt for your needs //else { // $wdtVar1 = "9999"; //Default to 9999 --> needed for Admin Context //} } add_action('wpdatatables_before_get_table_metadata', 'setVar1fromURL');
It doesn't work in back-end because the GET parameter is not set.
Remove this:
$varFromURL = $_GET['wdtVAR_1'];
And ask directly for GET:
if (isset($_GET['wdtVAR_1'])) { $wdtVar1 = $_GET['wdtVAR_1']; }
If the GET parameter isn't set per this code, VAR will be set to 9999, so you need to set the value you need, or remove it if you don't need the value for a GET parameter that isn't set.
Thank you Aleksander, this is very helpful. It does resolve 1 of 2 errors thrown. However I am still receiving the error related to Undefined index: table which still crashes the edit page disallowing me from editing my page.
Notice: Undefined index: table in /home2/objecto1/public_html/dataace/wp-content/plugins/wpdatatables/source/class.wpdatatable.php on line 1501
I'm wondering if I'm building my sql in such a way that it is throwing this error?
Example of most non normal sql on site:
SSELECT
NewCases_Geo,
DailyCases_Geo,
DailyCases.geoType,
DailyCases,
DailyCases_Change_p,
DailyCases_Change_n,
NewTests_Geo,
DailyTests_Geo,
DailyTests,
DailyTests_Change_p,
DailyTests_Change_n,
geoTypeLookup
FROM (
SELECT
CONCAT(Geo," (",ValueChange,")") AS DailyCases_Geo,
CONCAT(Geo," (",CurrentValue,")") AS NewCases_Geo,
Geo,
geoType,
CurrentValue AS DailyCases,
GREATEST(ValueChange, 0) AS DailyCases_Change_p,
ABS(LEAST(ValueChange, 0)) AS DailyCases_Change_n,
CASE SUBSTRING_INDEX(SUBSTRING_INDEX(uniqueID_Parent,'_',-2),'_',1)
WHEN 'State' THEN 'County'
WHEN 'Country' THEN 'State'
WHEN 'SubRegion' THEN 'Country'
WHEN 'Region' THEN 'SubRegion'
END AS geoTypeLookup
FROM COVID_Deltas
WHERE uniqueID_Parent = '%VAR1%'
AND Statistic = 'DailyCases'
AND PeriodDate = '1Day'
) AS DailyCases LEFT JOIN (
SELECT
Geo,
CONCAT(Geo," (",ValueChange,")") AS DailyTests_Geo,
CONCAT(Geo," (",CurrentValue,")") AS NewTests_Geo,
CurrentValue AS DailyTests,
GREATEST(ValueChange, 0) AS DailyTests_Change_p,
ABS(LEAST(ValueChange, 0)) AS DailyTests_Change_n
FROM COVID_Deltas
WHERE uniqueID_Parent = '%VAR1%'
AND Statistic = 'DailyTests'
AND PeriodDate = '1Day'
) AS DailyTests ON DailyTests.Geo = DailyCases.Geo
WHERE geoTypeLookup = geoType
function setVar1fromURL($tableID){
global $wdtVar1;
if (isset($_GET['wdtVAR_1'])) {
$wdtVar1 = $_GET['wdtVAR_1'];
}
else {
$wdtVar1 = "Wisconsin_State_UnitedStatesofAmerica";
}
}
add_action('wpdatatables_before_get_table_metadata', 'setVar1fromURL');
You're correct, it's because of the query. The parser is not able to parse it since it's too complicated.
You should try to create a MySQL view, which would hold the query. Unfortunately, you wouldn't be able to use the placeholders in the view, because it wouldn't work. You would need to create something like:
I have over 250,000 records that need to be returned and filtered for use in a chart however the table seems to take a long time or just does not load. Are there any recommended alternatives of doing this to generating a query then filtering the table based on a parameter value?
Digging further into the specific error I am seeing this in my logs:
More Info. The query runs in 1 second using a remote connection leading me to believe that TMS cannot handle this amount of records.
* Affected rows: 0 Found rows: 252,956 Warnings: 0 Duration for 1 query: 0.094 sec. (+ 1.000 sec. network) */
Hello Andrew
Sorry for the late response.
There are wpDataTables that work with millions of entries, so I don't believe that's the case.
You may want to increase the following:
upload_max_file size,
post_max_size,
max_execution_time
You should be able to find these values in your php.ini file or on your hosting panel. If you can't find it yourself please ask your hosting provider to provide it.
Other than that, you may want to look at your server's performance, hosting plan and the amount of other data you have on your pages. SQL query based tables (that don't pull ALL data onto a page) should work normally. Maybe a bit slower than non-server-side (smaller) tables, but work all-in-all.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Thanks Aleksandar. Those settings did not change the behavior. Is there any way I can pass a url paremeter into the where statement of the sql that is executed?
I know I can pass a url parameter to filter a dataset (that's what we are doing now) but that requires pulling all data then filtering.
I also know we can use placeholders which helps reduce the number of tables required but shifts us to have multiple pages instead of multiple tables which is really no gain on our part. Is there any way to pass a url parameter into a placeholder?
Hello Andrew
Using the built-in features of the plugin will not be able to do this, however there is a workaround using this hook:
wpdatatables_before_get_table_metadata( $tableId )
Although, there is a bug on our end, and I'll have to ask you to bear with me to get it working properly.
The action is called correctly when modifying a table in the backend, but if you have the same table rendered from the front-end the action is not called correctly.
If you change a global variable within this action - works in the backend, in the front-end the default value is used:
Unfortunately at the moment I can offer only this Hot-Fix with non desired-side-effects:
….SNIP
Here: class.wpdatatable.php
I hope this helps.
After a new update is released, you won't need to add the hot-fix, though.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Actually, Andrew,
We've had a new update is released, so you won't need to add the hot-fix.
The first part of the code goes into functions.php of your theme, while the other part of the code goes in ../wp-content/plugins/wpdatatables/source/class.wpdatatable.php. You can look for "public function fillFromData" in that file (should be around line 2722), and overwrite it.
Please save a copy of the original file, just in case something goes wrong, so you can restore it.
To be able to pass variables dynamically through URL, you should try something like this:
And in the browser:
That would return "10" and overwrite the original %VAR1% value.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Sorry, but I'm not understanding what goes into functions.php and what I'm overwriting in public function fillFromData from the screenshot below.
Hi again Andrew
This goes into functions.php:
Since the update was released, the second code is no longer necessary.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
I have upgraded and entered the following but I'm getting the error that no data is returned in the table. It appears that quotes that are need to select strings are not being added to %VAR1% parameter.
Rendered query: SELECT geo, parent FROM WorldStats WHERE Parent = Wisconsin LIMIT 10
MySQL said: Unknown column 'Wisconsin' in 'where clause'
Hi Andrew
Please provide me a temporary WP-admin login for your site where this happens, so we could log in and take a look ‘from the inside’ as that’s the most efficient way to see and resolve the issue. We do not interfere with any data or anything else except for the plugin (in case that’s a production version of the site), and of course we do not provide login data to third party.
You can write credentials here just check Private Reply so nobody can see them except us.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Hi Andrew.
Thank you for the credentials. However, they are not Administrator level credentials:
Can you please make that an admin role, and let me know?
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
My apologies. Updated them.
Hi Andrew.
Aleksandar is out of office today and I will help you out.
We check your website, and we notice that you only copy and paste code for hook that is the example from other customer needs which is the reason why your tables with placeholder VAR1 is not working. (attachment)
Another this that we found in table with id 88 is that you are using in condition Parent instead of parent. (attachment)
Please note that using hooks and filters requires some programming skills so you can adopt functionality for your needs.
If you want to filter the table with URL parameter you need to change hook example that Aleksandar provide to you.
Here are some advices in comments
Kind Regards,
Isidora Markovic
wpDataTables: FAQ | Facebook | Twitter | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Amelia demo sites | Docs
You can try our wpDataTables add-ons before purchase on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables
Hi again Andrew
Can you please share the code you've added with us?
As a reminder - you cannot simply copy the code. You need to adjust it for your needs. The code we've provided is just an example.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
My need is very simple for now. I have tried both of the below examples in functions.php. It appears that there is no element named table in the $parsedQuery['FROM'][0] in class.wpdatatable.php.
The functions error is likely happening in edit because there is no wdtVAR_1 here and it should instead load the default defined in Placeholders. It's as if isset is not acting as expected.
class.wpdatatable.php on line 1501
$tableName = $parsedQuery['FROM'][0]['table'];
functions.php on line 167
if (isset($varFromURL) && ($tableID == 93 || $tableID == 94 || $tableID == 99)) {
if (isset($varFromURL)) {
All additional Functions.php code
function setVar1fromURL($tableID){
global $wdtVar1;
//get URL parametar
$varFromURL = $_GET['wdtVAR_1'];
// Here is the condition that ask is that URL parameter set
// Code like this will be executed for every table with SQL query
// If you need for specific table then you will need to provide additional code
// for example you want to filter only table with id 1 then
// you will replace if (isset($varFromURL)) { with this if (isset($varFromURL && $tableID == 1)) {
//if (isset($varFromURL) && ($tableID == 93 || $tableID == 94 || $tableID == 99)) {
if (isset($varFromURL)) {
$wdtVar1 = $varFromURL; //Set VAR1 with parameter from URL
}
// if your condition is not true then this code below will be executed
// reason why your tables is not working because $wdtVar1 is always set to 9999
// if URL parameter wdtVAR_1 is not set
// if you don't need it you can remove it
// This is example for some other customer needs so you need to adopt for your needs
//else {
// $wdtVar1 = "9999"; //Default to 9999 --> needed for Admin Context
//}
}
add_action('wpdatatables_before_get_table_metadata', 'setVar1fromURL');
Hi again Andrew
It doesn't work in back-end because the GET parameter is not set.
Remove this:
And ask directly for GET:
If the GET parameter isn't set per this code, VAR will be set to 9999, so you need to set the value you need, or remove it if you don't need the value for a GET parameter that isn't set.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Thank you Aleksander, this is very helpful. It does resolve 1 of 2 errors thrown. However I am still receiving the error related to Undefined index: table which still crashes the edit page disallowing me from editing my page.
Notice: Undefined index: table in /home2/objecto1/public_html/dataace/wp-content/plugins/wpdatatables/source/class.wpdatatable.php on line 1501
I'm wondering if I'm building my sql in such a way that it is throwing this error?
Example of most non normal sql on site:
Hello again Andrew
You're correct, it's because of the query. The parser is not able to parse it since it's too complicated.
You should try to create a MySQL view, which would hold the query. Unfortunately, you wouldn't be able to use the placeholders in the view, because it wouldn't work. You would need to create something like:
SELECT * FROM view WHERE some_column= '%VAR1%'
Or something like that.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables