On your website you showcase WPKraken as an alliance partner that can help with a variety of Wordpress solutions. I engaged them to load up a CVS file with about 800,000 records (only 12 columns of data though).
Here was there reply and I'd like a response from wpdatables as it seems strange to me after reading the server side process documentation that wpdatatables has on your website, which mentions having a file with millions of rows using server side lookup.
After our investigation, we eventually concluded that the issue is within lack of compatibility in one of the Gravity Form addon - "Gravity Form - Populate Anything". While uploading the 500k DB records in your database is possible, the Gravity Form, when trying to set the Company Name, performs a query to the database, returning all records which basically breaks the script. The only option here is to edit the core files of the addon plugin, changing the way it operates on the backend, so not all 500k records are queried at once. This would take us about two weeks to finish and would cost $2000.
In fact, the Gravity Form plugin retrieves all records from a given table, because if you choose any table, it checks what values are in a given column to make it possible to set it in the third dropdown. We can see that you have quite an old version of this plugin, so it would be better to first update it and then check if it will work, otherwise you will have to limit it to e.g. 500 records when retrieving values from a given column, you will have limited functionality but it should work. It is tough to tell if this will work. We think the best solution is to again hire one of our developers for 8 hours to let him check and go through the code to see what could be done. The price is, as in previous one, $240.
Please let me know if any of those solutions will be suitable to you, or maybe would you like to consult the options with one of our WP Data Tables specialist before choosing the right one?
This seems to be a similar ticket as the one I replied to yesterday, so I'll post my response again with the difference of CSV vs Excel:
The CSV file would be too large to handle if you plan to create a table Linked to the CSV file, but if you were to import the CSV file, then you would be able to use wpDataTables for this.
The thing with importing the file is that once you import it, the wpDataTable will no longer be connected to the file - you'd get a manual table, saved in the database, which you can manipulate any way you want. To import a file this big, I believe you'll need to modify your php.ini file, located on your server, since the default upload limit is 2MB, and timeout time is set to 1 minute. So, you would need to increase those in order to use WordPress' upload feature to create an imported table.
Another way would be to save the file as CSV, and import it directly in your database. Then you can create an SQL Query based table which would pull the table from the database.
Since WP Kraken's response you've added doesn't mention CSV or Excel files, but instead Gravity Forms, I'm not sure I understand what you're trying to achieve.
Can you please post the question you forwarded to them, so I know what you're trying to do?
Thanks for the reply! Here is what I'm trying to do overall. I currently have several gravity forms that I use that are 'connected' to a wpdatatable. So for instance when a user selects a certain option on the form a number of data elements can be displayed real-time from the underlying wpdatatable. This works flawlessly and really allows a lot of data to be captured via lookup tables and be shown REAL TIME as the user is entering the data in the form. The wpdatatable and gravity form link if great.
What I want to do now is load a file that is about 500,000 rows long and about 12 columns wide. This dataset can be in either csv or excel format. From reading the documentation on the wpdatatable site I thought that a file of this size could be loaded into a mySQL database and use the server side processing option so it do could supply real-time data when a user is entering information into a gravity form.
That's it, it seems pretty simple to me except I don't know sql, etc. So basically if someone enters in a Company Name in a Gravity Form, the form will cross reference the database and if it finds that Company Name it will pull and display fields of information from the large file.
I hope that makes sense but let me know if you need more information.
A file this big can definitely be handled by wpDataTables if the generated table is an SQL query based table.
However, the main issue here is cross-referencing it with the form. You see, Gravity Forms stores its data into several database tables:
wpDataTables' add-on basically constructs a query in back-end, and displays results from that form. That query is rather complex, and is being calculated on several factors (form fields' type, relation, size, etc.) and it's not a unique query for all forms. If it was easy to pull data from Gravity Forms, we wouldn't have even created the add-on, but simply show an example of the query to anyone wanting to display form's entries in tabular form.
When you import that file, it will be saved in the database exactly how you see it in Excel, so one table with 12 columns. Adding that to the form is complicated because you would need to associate the data from that table to those 7 Gravity Forms tables and its columns. Gravity's database tables store their data in a serialized array (some columns, not all), and doing that programmatically is difficult to say the least.
I have a suggestion for you, but I never used it like this, so I can't guarantee it will work. It's worth a shot, though.
Download Gravity WPDB plugin. It reads the form and stores its data into one database table, so you would be able to manipulate that data easier than with those 7 Gravity database tables. I don't know if the plugin works vice-versa, that is - if it can update form's entries with data entered directly in that newly created table.
I wouldn't be able to help you with this since that falls under customization, and we don't have time to accept any custom work at the moment.
On your website you showcase WPKraken as an alliance partner that can help with a variety of Wordpress solutions. I engaged them to load up a CVS file with about 800,000 records (only 12 columns of data though).
Here was there reply and I'd like a response from wpdatables as it seems strange to me after reading the server side process documentation that wpdatatables has on your website, which mentions having a file with millions of rows using server side lookup.
Please reply ASAP, thanks:
=========================================================
After our investigation, we eventually concluded that the issue is within lack of compatibility in one of the Gravity Form addon - "Gravity Form - Populate Anything". While uploading the 500k DB records in your database is possible, the Gravity Form, when trying to set the Company Name, performs a query to the database, returning all records which basically breaks the script. The only option here is to edit the core files of the addon plugin, changing the way it operates on the backend, so not all 500k records are queried at once. This would take us about two weeks to finish and would cost $2000.
In fact, the Gravity Form plugin retrieves all records from a given table, because if you choose any table, it checks what values are in a given column to make it possible to set it in the third dropdown. We can see that you have quite an old version of this plugin, so it would be better to first update it and then check if it will work, otherwise you will have to limit it to e.g. 500 records when retrieving values from a given column, you will have limited functionality but it should work. It is tough to tell if this will work. We think the best solution is to again hire one of our developers for 8 hours to let him check and go through the code to see what could be done. The price is, as in previous one, $240.
Please let me know if any of those solutions will be suitable to you, or maybe would you like to consult the options with one of our WP Data Tables specialist before choosing the right one?
Hello again Tony
This seems to be a similar ticket as the one I replied to yesterday, so I'll post my response again with the difference of CSV vs Excel:
The CSV file would be too large to handle if you plan to create a table Linked to the CSV file, but if you were to import the CSV file, then you would be able to use wpDataTables for this.
The thing with importing the file is that once you import it, the wpDataTable will no longer be connected to the file - you'd get a manual table, saved in the database, which you can manipulate any way you want. To import a file this big, I believe you'll need to modify your php.ini file, located on your server, since the default upload limit is 2MB, and timeout time is set to 1 minute. So, you would need to increase those in order to use WordPress' upload feature to create an imported table.
Another way would be to save the file as CSV, and import it directly in your database. Then you can create an SQL Query based table which would pull the table from the database.
Since WP Kraken's response you've added doesn't mention CSV or Excel files, but instead Gravity Forms, I'm not sure I understand what you're trying to achieve.
Can you please post the question you forwarded to them, so I know what you're trying to do?
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables
Alek,
Thanks for the reply! Here is what I'm trying to do overall. I currently have several gravity forms that I use that are 'connected' to a wpdatatable. So for instance when a user selects a certain option on the form a number of data elements can be displayed real-time from the underlying wpdatatable. This works flawlessly and really allows a lot of data to be captured via lookup tables and be shown REAL TIME as the user is entering the data in the form. The wpdatatable and gravity form link if great.
What I want to do now is load a file that is about 500,000 rows long and about 12 columns wide. This dataset can be in either csv or excel format. From reading the documentation on the wpdatatable site I thought that a file of this size could be loaded into a mySQL database and use the server side processing option so it do could supply real-time data when a user is entering information into a gravity form.
That's it, it seems pretty simple to me except I don't know sql, etc. So basically if someone enters in a Company Name in a Gravity Form, the form will cross reference the database and if it finds that Company Name it will pull and display fields of information from the large file.
I hope that makes sense but let me know if you need more information.
Hey Tony
That makes sense, thanks for the explanation.
A file this big can definitely be handled by wpDataTables if the generated table is an SQL query based table.
However, the main issue here is cross-referencing it with the form. You see, Gravity Forms stores its data into several database tables:
wpDataTables' add-on basically constructs a query in back-end, and displays results from that form. That query is rather complex, and is being calculated on several factors (form fields' type, relation, size, etc.) and it's not a unique query for all forms. If it was easy to pull data from Gravity Forms, we wouldn't have even created the add-on, but simply show an example of the query to anyone wanting to display form's entries in tabular form.
When you import that file, it will be saved in the database exactly how you see it in Excel, so one table with 12 columns. Adding that to the form is complicated because you would need to associate the data from that table to those 7 Gravity Forms tables and its columns. Gravity's database tables store their data in a serialized array (some columns, not all), and doing that programmatically is difficult to say the least.
I have a suggestion for you, but I never used it like this, so I can't guarantee it will work. It's worth a shot, though.
Download Gravity WPDB plugin. It reads the form and stores its data into one database table, so you would be able to manipulate that data easier than with those 7 Gravity database tables. I don't know if the plugin works vice-versa, that is - if it can update form's entries with data entered directly in that newly created table.
I wouldn't be able to help you with this since that falls under customization, and we don't have time to accept any custom work at the moment.
Kind Regards,
Aleksandar Vuković
[email protected]
Rate my support
wpDataTables: FAQ | Facebook | Twitter | Instagram | Front-end and back-end demo | Docs
Amelia: FAQ | Facebook | Twitter | Instagram | Amelia demo sites | Docs | Discord Community
You can try wpDataTables add-ons before purchasing on these sandbox sites:
Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables