Hi, Andrew -

Well, I had some free time recently while getting acquainted with and enjoying my newborn grandson, so I created a non-visual object (NVO) that implements the ASCII Folding Filter functionality. It was developed using PB 2017. A Zip file containing the pbl, workspace and target is attached.

The app is a very simple, single-window application that illustrates how the of_FoldToASCII function in the NVO can be used.

Please Note: My conversion of the Lucene.Net class definition is not designed or intended for mass filtering of large amounts of Unicode text.

It performs the same translation coded in the Apache/Lucene.Net class. You supply an input string, a starting position and the length (number of characters) to be examined, similar arguments as the PowerScript Mid function. The translated input string is returned. There are two alternative versions of the of_FoldToASCII function (overrides) with fewer arguments.

I hope you find the NVO and the sample application helpful. Enjoy!

John

Greetings, Andrew -

In order to accomplish this, I believe you need two things:

1. A means of replacing diacritics in Unicode strings with their plain ASCII-127 "equivalent" characters. Perhaps something akin to the java ASCIIFoldingFilter method in the org.apache.lucene.analysis package.

2. If the information in the DBMS holds string information that contains diacritics that you wish to search/find, then you would likely need or want a version of this information in the DBMS that already has the diacritics replaced, for performance reasons.

You probably would only want to perform this kind of search when the user selects a special "Advanced Search" option.

It would be terrific if someone had a PB-compatible version of the ASCIIFoldingFilter function and was able to share it, but I'm not aware of anyone publishing such a thing. If you can find/use source code that does this or similar, it might be possible to port the code to PowerScript or to C++ and make it available to PowerScript via PBNI.

Good luck!

Regards, John

I don't think there's any existing PB functionality that does this, you'd have to program it yourself. Step one would probably be the creation of a lookup table containing every possible variant of all letters and their corresponding "standard" ASCII character. Then you can use this table to convert both the search string and the database results to pure ASCII strings and compare the results.

What method/tool are you using to search?

Hi Andrew,

yes, I know about this and other issues like ß = ss.

I think it's a feature :-)