Okay
  Public Ticket #3830951
ampersand in data, problem when searching and filtering
Open

Comments

  •  1
    Alex started the conversation

    Hello,

    let's say we have a column which has telecommunication company names in it, so one of the values is "AT&T".

    Global search fails to find any AT&T rows despite they exist in the table. Not necessarily a big problem in my use case, but also the column filter fails to return desired results here and this is not good at all.

    I didn't test other methods like powerful filters yet, but I suspect the problem occurs in any search and filter.

    Could you please conduct some tests and hopefully provide a fix or a workaround (replacing the brand as "ATT" is not an option). Thank you.

    Kind regards, Alex

  •  1,888
    Miloš replied

    Hi Alex,

    Our developers have identified the root of this bug,

    that does not correctly render special characters such as the percentage sign for filtering in selectbox and/or multiselectbox filters.

    Note that this bug only happens to non-server-side tables. It does not happen to server-side processing tables ( Manual tables and SQL Query tables with server-side enabled).

     

    They are working on building a permanent fix for this,

    but it is a very difficult bug,  so it will take some time until they test everything to ensure that the new fix does not negatively affect any other plugin functionalities before they add it to the plugin's core.

     

    They will do their best to make it as soon as possible, but i can't promise a realistic ETA on it.

    You can certainly follow our changeLog page if you'd like ( it is also available in the plugin dashboard), where we state any changes/new features/bug fixes during updates;

    and our newsletter, so you're informed about new features, bug fixes, freebies, etc.

    We will make a patch Note about that fix there, when it is finished/added to the plugin core.

     

    In the meantime, here is a temporary JS fix that you can copy to overwrite the file on your FTP in the plugin files.

    ( You can either use FTP or your Hosting's File Manager)

    The path to overwrite the file should be :

    ../wp-content/plugins/wpdatatables/assets/js/wpdatatables/wdt.columnFilter.js

    and please make sure to disable 'Use minified wpDataTables Javascript' in the main plugin settings/Custom JS and CSS:

    6577636354.png

    Until our developers implement a fix to the plugin's core,

    you would have to repeat this "overwrite" step each time you update the plugin.

    This is not a perfect fix - It seems to be only working for SelectBox, but not for MultiSelectbox and it seems to work only for some specific characters, not for all of them, that is why it is still a work in progress.

    Let me know if you apply this fix, if it works correctly for you, please.


    2. Another workaround to avoid the bug, could be, if possible, for you to switch to server-side processing tables.
    It depends on which table type you use at the moment.
    For example, if you have the data on Excel/CSV/ or Google Sheet, you can import it to make a server-side/editable table.

    Later on, you can edit this table from our Plugin and it can also be updated from file upload.

    I hope that helps.

    Thank you.

    Attached files:  wdt.columnFilter 6.1.zip

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables

  •  1
    Alex replied

    Hi Miloš,

    thank you for the reply and workaround.

    However, this is a manual table. So probably a different bug.

    Should be easy to test, just try adding AT&T or similar brand into a string column and see what happens with the searches/filters.

    I now found global search and filtering do return correct results when the ampersand sign is literally replaced with the corresponding HTML character entity when typing (ampersand, "amp" and semicolon). This actually seems to be happening when the filter type for the column is set to selectbox, the HTML entity is visible in the selectbox... doesn't look pretty but works.

    So there should be some character replacement happening in places in order to get this working the way it should.

    I guess this existing temporary JS fix would not make a difference in this case..?

    Kind regards, Alex

  •  1,888
    Miloš replied

    Hi Alex,

    You are right, thank you for reporting this.

    This symbol is behaving differently; it is the opposite behaviour as that known bug we have with some other special characters.

    The bug we know is for some special characters not working when the column filter type is set as dropdown (Selectbox), and that works in a Text filter.

    Additionally, that bug happens only to non-server-side tables.

    Here, as you pointed out, the & symbol gets stored in the Database value as "&".
    It actually works in dropdown filtering, while it does not work in a Text filter and on global table search.

    It seems we isolated more details.


    1. If we use our Standard Editor, such as "One Line String", this saves the value as & in the Database, so it we search a text filter or global table search as AT&T, no results.

    2. But if we switch to Excel-Like Editor, now it saves the value correctly without the ampersand, so it works in all filters and table searches as AT&T.
    6104672284.png

    8705042005.png

    6173459189.png
    You can download my Video to see this quick demonstration in more detail.

    As per my example, the table only gives the results for the rows I edited via the Excel-Like version, while the Standard "one line/multi line String" editor adds that ampersand.

    I am passing this report to our QA Team to investigate how the standard editor saves this symbol, as it affects filtering and search functionality.
    They will add it to the Development Team's queue for fixing.

    Our developers will do their best to implement a fix as soon as possible.

     We don't have an ETA on when the fix will be available, but please keep a lookout on our wpDataTables changeLog.

    It will be stated there during updates which bug fixes have been implemented at each update.
    If they create a fix in between updates, I will make sure to send it to you.

    For the time being, possible workarounds can be  :
    - To use Excel-Like Editor for editing, if possible for your table, then it stores the & without the additional ampersand.

    - Or, use the dropdown filtering instead of Text filter type for these kinds of values.

    I hope that helps.

    Kind Regards, 

    Miloš Jovanović
    [email protected]

    Rate my support

    Try our FREE mapping plugin! MapSVG - easy Google maps, interactive SVG maps and floor plans, choropleth maps and much more - https://wordpress.org/plugins/mapsvg-lite-interactive-vector-maps/

    wpDataTables: FAQ | Facebook | Twitter | InstagramFront-end and back-end demo | Docs

    Amelia: FAQ | Facebook | Twitter | InstagramAmelia demo sites | Docs | Discord Community

    You can try wpDataTables add-ons before purchasing on these sandbox sites:

    Powerful Filters | Gravity Forms Integration for wpDataTables | Formidable Forms Integration for wpDataTables | Master-Detail Tables