Hi Mark

li_file_handle = FileOpen(ls_path_filename, StreamMode!, Read!, LockReadWrite!, Append!, EncodingANSI!)
ll_chars_read = FileReadEx(li_file_handle,lblb_file_contents)
ls_contents_as_utf8 = String(lblb_file_contents, EncodingUTF8!)

It works fine and the text in txt file stored in DB column accurately

thank you

Regards ...

Ahmed

Hi Ahmed,

Daryl's speculation on the issue being related to encodings is correct. As you can see in the picture below, when I opened the text file you provided, it shows as UTF-8 encoding, but PowerBuilder requires UTF-8 with BOM in order to successfully open it with FileOpen() and EncodingUTF8! Actually, if you manually save the file but change it to UTF-8 with BOM (with Notepad, Notepad++ etc.) and then try to process it I suspect it will work. You may wish to see if you can control the encoding that Apache Tika uses when creating the text file.

If you can't control the encoding there are a couple of options I can think of:

1. Once Apache Tika creates the file, open it and manually add the BOM encoding bytes (EF BB BF) to the start of the file before you try and process it, or
2. The following code will allow you to get around the absence of the BOM bytes:

li_file_handle = FileOpen(ls_path_filename, StreamMode!, Read!, LockReadWrite!, Append!, EncodingANSI!)
ll_chars_read = FileReadEx(li_file_handle,lblb_file_contents)
ls_contents_as_utf8 = String(lblb_file_contents, EncodingUTF8!)

The above 3 lines of code are basically opening the file in stream mode with ANSI encoding; then reading the contents of the file into a Blob variable; then converting the blob contents to a string using UTF-8 encoding...now you can process what's contained in ls_contents_as_utf8.

HTH...regards,

Mark

Hi Daryl

zip file attached for docx,txt,message and code

also rar file attached

Hi Ahmed,

Try adding 'CMD /c in front of your first double-quote and add a single quote after your last double-quote. So it looks like the following:

Run('CMD /c "java -jar tika-app-2.0.0.jar e:\a1.docx > e:\abc.txt 2> e:\err.err"')

This should open a command window, execute what you have provided in double-quotes and then close.

HTH...regards,

Mark

Hi all

I've successfully used apache tika to extract the arabic  text from the CMD, how would I do it from the powerbuilder, can i use run ??

And how could I use the output text to put it in the required column automatically (without user interaction){KomentoLock}.

Regards Ahmed

Hi, Ahmed -

As a test, try manually editing a Word document using Notepad. You will not see the textual content of the document.

This illustrates what Daryl describes. The text in a Word document is not in a "clear text" format... you must either use Word to search the contents of a Word document or you must first extract the text content and then search that.

The follow-up link that Chris has provided describes searching "character-based" text, NOT text in Word documents. Once you have the textual content of a Word document (Daryl has described one technique for accomplishing this), then you may be able to provide search functionality.

Regards,
John

Hi Ahmed, I think the issue you will have searching for text in the Word documents is that .docx files are actually binary zip files which contain a bunch of xml files, so you won't be able to search for the text inside them unless you (at least) decompress them first and then search in the relevant xml files.  An alternative is to extract the text only from the .docx and store that in a separate varchar(max) column which you could use for searching.

We use Apache Solr for our full text searching solution and we use Apache Tika to extract the text from .docx files to load into Solr.

Hi Ahmed;

   This feature is very DBMS vendor dependent.

For example: http://docs.microsoft.com/en-us/azure/storage/blobs/storage-manage-find-blobs?tabs=azure-portal

Regards ... Chris